{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 共享单车骑行量预测案例——线性回归分析\n",
    "\n",
    "在这个案例中，我们将利用Capital Bikeshare （美国Washington, D.C.的一个共享单车公司）提供的自行车数据上训练和测试一个模型，并对模型的性能和预测能力进行测试。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "回归模型的评价指标  \n",
    "1、开方均方误差（Rooted Mean Squared Error，RMSE）  \n",
    "2、平均绝对误差（Mean Absolute Error，MAE）  \n",
    "3、绝对误差中值  \n",
    "4、平均平方log误差（Mean Squared Logarithmic Error, MSLE）:当 𝑦呈指数增长时可以使用（如计数、一年的平均销量…）  \n",
    "\n",
    "•Scikit-Learn中有三种评价模型性能的方式：   \n",
    "• estimator的score方法：每个学习器都有score方法，提供一个 缺省的评估方法（回归为𝑅>分数）。   \n",
    "• Metric： metrics模块实现了一些函数，用来评估预测误差。   \n",
    "Scikit-Learn中metrics的回归评价指标  \n",
    "sklearn.metrics  \n",
    "metrics.explained_variance_score(y_true, y_pred) ：Explained variance regression score function   \n",
    "metrics.mean_absolute_error(y_true, y_pred)：Mean absolute error regression loss   \n",
    "metrics.mean_squared_error(y_true, y_pred[, …])：Mean squared error regression loss   \n",
    "metrics.mean_squared_log_error(y_true, y_pred)：Mean squared logarithmic error regression loss   \n",
    "metrics.median_absolute_error(y_true, y_pred)：Median absolute error regression loss   \n",
    "metrics.r2_score(y_true, y_pred[, …])： R^2 (coefficient of determination) regression score   \n",
    "• Scoring参数：使用交叉验证评估模型的工具有Scoring 参数。  \n",
    "\n",
    "\n",
    " 5、𝑅方分数  \n",
    " 6、已解释的方差分数（Explained variance score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1、导入必要的工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np  # 矩阵操作\n",
    "import pandas as pd # SQL数据处理\n",
    "\n",
    "#from sklearn.metrics import r2_score  #评价回归预测模型的性能\n",
    "from sklearn.metrics import mean_squared_error  #评价回归预测模型的性能，按照作业要求RMSE\n",
    "\n",
    "import matplotlib.pyplot as plt   #画图\n",
    "import seaborn as sns\n",
    "\n",
    "# 图形出现在Notebook里而不是新窗口\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 读取数据\n",
    "做完特征工程后的数据，请先运行2. FE_BikeSharing.ipynb，得到文件FE_day.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>yr</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.355170</td>\n",
       "      <td>0.373517</td>\n",
       "      <td>0.828620</td>\n",
       "      <td>0.284606</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.379232</td>\n",
       "      <td>0.360541</td>\n",
       "      <td>0.715771</td>\n",
       "      <td>0.466215</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.171000</td>\n",
       "      <td>0.144830</td>\n",
       "      <td>0.449638</td>\n",
       "      <td>0.465740</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.175530</td>\n",
       "      <td>0.174649</td>\n",
       "      <td>0.607131</td>\n",
       "      <td>0.284297</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.209120</td>\n",
       "      <td>0.197158</td>\n",
       "      <td>0.449313</td>\n",
       "      <td>0.339143</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   instant  season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  \\\n",
       "0        1         1         0         0         0       1       0       0   \n",
       "1        2         1         0         0         0       1       0       0   \n",
       "2        3         1         0         0         0       1       0       0   \n",
       "3        4         1         0         0         0       1       0       0   \n",
       "4        5         1         0         0         0       1       0       0   \n",
       "\n",
       "   mnth_4  mnth_5  ...  weekday_5  weekday_6      temp     atemp       hum  \\\n",
       "0       0       0  ...          0          1  0.355170  0.373517  0.828620   \n",
       "1       0       0  ...          0          0  0.379232  0.360541  0.715771   \n",
       "2       0       0  ...          0          0  0.171000  0.144830  0.449638   \n",
       "3       0       0  ...          0          0  0.175530  0.174649  0.607131   \n",
       "4       0       0  ...          0          0  0.209120  0.197158  0.449313   \n",
       "\n",
       "   windspeed  holiday  workingday  yr   cnt  \n",
       "0   0.284606        0           0   0   985  \n",
       "1   0.466215        0           0   0   801  \n",
       "2   0.465740        0           1   0  1349  \n",
       "3   0.284297        0           1   0  1562  \n",
       "4   0.339143        0           1   0  1600  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# path to where the data lies\n",
    "#dpath = './data/'\n",
    "df = pd.read_csv(\"FE_day.csv\")\n",
    "\n",
    "#通过观察前5行，了解数据每列（特征）的概况\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  数据基本信息\n",
    "样本数目、特征维数\n",
    "每个特征的类型、空值样本的数目、数据类型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 731 entries, 0 to 730\n",
      "Data columns (total 35 columns):\n",
      "instant         731 non-null int64\n",
      "season_1        731 non-null int64\n",
      "season_2        731 non-null int64\n",
      "season_3        731 non-null int64\n",
      "season_4        731 non-null int64\n",
      "mnth_1          731 non-null int64\n",
      "mnth_2          731 non-null int64\n",
      "mnth_3          731 non-null int64\n",
      "mnth_4          731 non-null int64\n",
      "mnth_5          731 non-null int64\n",
      "mnth_6          731 non-null int64\n",
      "mnth_7          731 non-null int64\n",
      "mnth_8          731 non-null int64\n",
      "mnth_9          731 non-null int64\n",
      "mnth_10         731 non-null int64\n",
      "mnth_11         731 non-null int64\n",
      "mnth_12         731 non-null int64\n",
      "weathersit_1    731 non-null int64\n",
      "weathersit_2    731 non-null int64\n",
      "weathersit_3    731 non-null int64\n",
      "weekday_0       731 non-null int64\n",
      "weekday_1       731 non-null int64\n",
      "weekday_2       731 non-null int64\n",
      "weekday_3       731 non-null int64\n",
      "weekday_4       731 non-null int64\n",
      "weekday_5       731 non-null int64\n",
      "weekday_6       731 non-null int64\n",
      "temp            731 non-null float64\n",
      "atemp           731 non-null float64\n",
      "hum             731 non-null float64\n",
      "windspeed       731 non-null float64\n",
      "holiday         731 non-null int64\n",
      "workingday      731 non-null int64\n",
      "yr              731 non-null int64\n",
      "cnt             731 non-null int64\n",
      "dtypes: float64(4), int64(31)\n",
      "memory usage: 200.0 KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 特征两两之间的相关性     \n",
    "查看特征两两之间的相关系数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature1</th>\n",
       "      <th>featute2</th>\n",
       "      <th>corr</th>\n",
       "      <th>abs_corr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>540</th>\n",
       "      <td>temp</td>\n",
       "      <td>atemp</td>\n",
       "      <td>0.991702</td>\n",
       "      <td>0.991702</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>425</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>-0.938964</td>\n",
       "      <td>0.938964</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>instant</td>\n",
       "      <td>yr</td>\n",
       "      <td>0.866025</td>\n",
       "      <td>0.866025</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>119</th>\n",
       "      <td>season_3</td>\n",
       "      <td>temp</td>\n",
       "      <td>0.678471</td>\n",
       "      <td>0.678471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>120</th>\n",
       "      <td>season_3</td>\n",
       "      <td>atemp</td>\n",
       "      <td>0.655956</td>\n",
       "      <td>0.655956</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59</th>\n",
       "      <td>season_1</td>\n",
       "      <td>atemp</td>\n",
       "      <td>-0.625061</td>\n",
       "      <td>0.625061</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>season_1</td>\n",
       "      <td>temp</td>\n",
       "      <td>-0.619800</td>\n",
       "      <td>0.619800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>481</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>workingday</td>\n",
       "      <td>-0.602541</td>\n",
       "      <td>0.602541</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>538</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>workingday</td>\n",
       "      <td>-0.602541</td>\n",
       "      <td>0.602541</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>436</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>hum</td>\n",
       "      <td>-0.575317</td>\n",
       "      <td>0.575317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>season_4</td>\n",
       "      <td>mnth_10</td>\n",
       "      <td>0.536581</td>\n",
       "      <td>0.536581</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>season_1</td>\n",
       "      <td>mnth_1</td>\n",
       "      <td>0.530670</td>\n",
       "      <td>0.530670</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>136</th>\n",
       "      <td>season_4</td>\n",
       "      <td>mnth_11</td>\n",
       "      <td>0.527068</td>\n",
       "      <td>0.527068</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71</th>\n",
       "      <td>season_2</td>\n",
       "      <td>mnth_5</td>\n",
       "      <td>0.524889</td>\n",
       "      <td>0.524889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>104</th>\n",
       "      <td>season_3</td>\n",
       "      <td>mnth_8</td>\n",
       "      <td>0.517373</td>\n",
       "      <td>0.517373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103</th>\n",
       "      <td>season_3</td>\n",
       "      <td>mnth_7</td>\n",
       "      <td>0.517373</td>\n",
       "      <td>0.517373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>season_2</td>\n",
       "      <td>mnth_4</td>\n",
       "      <td>0.515584</td>\n",
       "      <td>0.515584</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>season_1</td>\n",
       "      <td>mnth_2</td>\n",
       "      <td>0.506932</td>\n",
       "      <td>0.506932</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         feature1      featute2      corr  abs_corr\n",
       "540          temp         atemp  0.991702  0.991702\n",
       "425  weathersit_1  weathersit_2 -0.938964  0.938964\n",
       "32        instant            yr  0.866025  0.866025\n",
       "119      season_3          temp  0.678471  0.678471\n",
       "120      season_3         atemp  0.655956  0.655956\n",
       "59       season_1         atemp -0.625061  0.625061\n",
       "58       season_1          temp -0.619800  0.619800\n",
       "481     weekday_0    workingday -0.602541  0.602541\n",
       "538     weekday_6    workingday -0.602541  0.602541\n",
       "436  weathersit_1           hum -0.575317  0.575317\n",
       "135      season_4       mnth_10  0.536581  0.536581\n",
       "36       season_1        mnth_1  0.530670  0.530670\n",
       "136      season_4       mnth_11  0.527068  0.527068\n",
       "71       season_2        mnth_5  0.524889  0.524889\n",
       "104      season_3        mnth_8  0.517373  0.517373\n",
       "103      season_3        mnth_7  0.517373  0.517373\n",
       "70       season_2        mnth_4  0.515584  0.515584\n",
       "37       season_1        mnth_2  0.506932  0.506932"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_corr = df.corr()\n",
    "data_corr_features = data_corr.drop(['cnt'], axis = 1).drop(['cnt'], axis = 0) #去掉目标变量的相关系数矩阵\n",
    "size = data_corr_features.shape[0] #特征数量\n",
    "\n",
    "corr_list = []\n",
    "for i in range(0,size):\n",
    "    for j in range(i+1,size):\n",
    "        corr_list.append([data_corr_features.index[i],data_corr_features.columns[j],data_corr_features.iloc[i,j],abs(data_corr_features.iloc[i,j])])\n",
    "        \n",
    "corr_features_df = pd.DataFrame(corr_list,columns=['feature1','featute2','corr','abs_corr'])\n",
    "corr_features_df[corr_features_df['abs_corr']>0.5].sort_values(by='abs_corr',ascending=False) #按照相关系数绝对值排序"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "temp和atemp相关系数达到0.99以上为正常现象，从特征含义就可看出，使用OLS考虑去除一个特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    710\n",
       "1     21\n",
       "Name: weathersit_3, dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['weathersit_3'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "weathersit_1 和 weathersit_2的相关系数高达0.938964，从上面weathersit_3为1的情况只有21样本，占比太少，所以weathersit_1 和 weathersit_2的相关系数才会这么高"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    366.000000\n",
       "mean     548.500000\n",
       "std      105.799338\n",
       "min      366.000000\n",
       "25%      457.250000\n",
       "50%      548.500000\n",
       "75%      639.750000\n",
       "max      731.000000\n",
       "Name: instant, dtype: float64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df['yr']==1]['instant'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    365.000000\n",
       "mean     183.000000\n",
       "std      105.510663\n",
       "min        1.000000\n",
       "25%       92.000000\n",
       "50%      183.000000\n",
       "75%      274.000000\n",
       "max      365.000000\n",
       "Name: instant, dtype: float64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df['yr']==0]['instant'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面的结果看证明所想，instant 1到365都是2011年的记录数据，instant 366到731都是2012年的记录数据，所以其instant和yr的相关系数高达0.866。\n",
    "instant记录号不作为特征使用，要在后面的数据集中去除\n",
    "yr只有2011和2012两个值，这个不能作为特征使用，预测一般都是预测未来的目标值，可以考虑其他与时间有关的特征，例如运营年限等"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 多重共线性    \n",
    "如果模型自变量之间存在严重的多重共线性的话\n",
    "+ 导致OLS估计量可能无效  \n",
    "+ 增大OLS估计量的方差  \n",
    "+ 变量的显著性检验将失去意义  \n",
    "+ 模型缺乏稳定性  \n",
    "多重共线性检验可以使用方差膨胀因子VIF来鉴定，如果VIF大于10，则说明变量存在多重共线性。\n",
    "如果存在多重共线性可能增大样本量（这个很多情况不太现实）、基于经验删除特征、逐步回归法、主成分分析、重新选择模型（如岭回归）\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 特征与目标变量之间的相关性    \n",
    "查看特征与目标变量之间的相关系数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>corr</th>\n",
       "      <th>abs_corr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>atemp</th>\n",
       "      <td>atemp</td>\n",
       "      <td>0.631066</td>\n",
       "      <td>0.631066</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>instant</th>\n",
       "      <td>instant</td>\n",
       "      <td>0.628830</td>\n",
       "      <td>0.628830</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>temp</th>\n",
       "      <td>temp</td>\n",
       "      <td>0.627494</td>\n",
       "      <td>0.627494</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>yr</th>\n",
       "      <td>yr</td>\n",
       "      <td>0.566710</td>\n",
       "      <td>0.566710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>season_1</th>\n",
       "      <td>season_1</td>\n",
       "      <td>-0.563094</td>\n",
       "      <td>0.563094</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mnth_1</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-0.366090</td>\n",
       "      <td>0.366090</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>season_3</th>\n",
       "      <td>season_3</td>\n",
       "      <td>0.346487</td>\n",
       "      <td>0.346487</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mnth_2</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-0.277764</td>\n",
       "      <td>0.277764</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weathersit_1</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>0.252870</td>\n",
       "      <td>0.252870</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weathersit_3</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-0.239958</td>\n",
       "      <td>0.239958</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>windspeed</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-0.234545</td>\n",
       "      <td>0.234545</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mnth_6</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>0.195866</td>\n",
       "      <td>0.195866</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mnth_9</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>0.194963</td>\n",
       "      <td>0.194963</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mnth_8</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>0.182426</td>\n",
       "      <td>0.182426</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mnth_12</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>-0.173065</td>\n",
       "      <td>0.173065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weathersit_2</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>-0.172879</td>\n",
       "      <td>0.172879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mnth_7</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>0.166584</td>\n",
       "      <td>0.166584</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>season_2</th>\n",
       "      <td>season_2</td>\n",
       "      <td>0.146197</td>\n",
       "      <td>0.146197</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mnth_5</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>0.132947</td>\n",
       "      <td>0.132947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mnth_3</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>-0.127705</td>\n",
       "      <td>0.127705</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mnth_10</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>0.109273</td>\n",
       "      <td>0.109273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>hum</th>\n",
       "      <td>hum</td>\n",
       "      <td>-0.100659</td>\n",
       "      <td>0.100659</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>holiday</th>\n",
       "      <td>holiday</td>\n",
       "      <td>-0.068348</td>\n",
       "      <td>0.068348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>season_4</th>\n",
       "      <td>season_4</td>\n",
       "      <td>0.065593</td>\n",
       "      <td>0.065593</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>workingday</th>\n",
       "      <td>workingday</td>\n",
       "      <td>0.061156</td>\n",
       "      <td>0.061156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weekday_0</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>-0.058288</td>\n",
       "      <td>0.058288</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mnth_11</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>-0.039723</td>\n",
       "      <td>0.039723</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weekday_5</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>0.039118</td>\n",
       "      <td>0.039118</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weekday_1</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>-0.035166</td>\n",
       "      <td>0.035166</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weekday_4</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>0.034273</td>\n",
       "      <td>0.034273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weekday_6</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>0.009773</td>\n",
       "      <td>0.009773</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weekday_3</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>0.009297</td>\n",
       "      <td>0.009297</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mnth_4</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>-0.003004</td>\n",
       "      <td>0.003004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weekday_2</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>0.001328</td>\n",
       "      <td>0.001328</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   feature      corr  abs_corr\n",
       "atemp                atemp  0.631066  0.631066\n",
       "instant            instant  0.628830  0.628830\n",
       "temp                  temp  0.627494  0.627494\n",
       "yr                      yr  0.566710  0.566710\n",
       "season_1          season_1 -0.563094  0.563094\n",
       "mnth_1              mnth_1 -0.366090  0.366090\n",
       "season_3          season_3  0.346487  0.346487\n",
       "mnth_2              mnth_2 -0.277764  0.277764\n",
       "weathersit_1  weathersit_1  0.252870  0.252870\n",
       "weathersit_3  weathersit_3 -0.239958  0.239958\n",
       "windspeed        windspeed -0.234545  0.234545\n",
       "mnth_6              mnth_6  0.195866  0.195866\n",
       "mnth_9              mnth_9  0.194963  0.194963\n",
       "mnth_8              mnth_8  0.182426  0.182426\n",
       "mnth_12            mnth_12 -0.173065  0.173065\n",
       "weathersit_2  weathersit_2 -0.172879  0.172879\n",
       "mnth_7              mnth_7  0.166584  0.166584\n",
       "season_2          season_2  0.146197  0.146197\n",
       "mnth_5              mnth_5  0.132947  0.132947\n",
       "mnth_3              mnth_3 -0.127705  0.127705\n",
       "mnth_10            mnth_10  0.109273  0.109273\n",
       "hum                    hum -0.100659  0.100659\n",
       "holiday            holiday -0.068348  0.068348\n",
       "season_4          season_4  0.065593  0.065593\n",
       "workingday      workingday  0.061156  0.061156\n",
       "weekday_0        weekday_0 -0.058288  0.058288\n",
       "mnth_11            mnth_11 -0.039723  0.039723\n",
       "weekday_5        weekday_5  0.039118  0.039118\n",
       "weekday_1        weekday_1 -0.035166  0.035166\n",
       "weekday_4        weekday_4  0.034273  0.034273\n",
       "weekday_6        weekday_6  0.009773  0.009773\n",
       "weekday_3        weekday_3  0.009297  0.009297\n",
       "mnth_4              mnth_4 -0.003004  0.003004\n",
       "weekday_2        weekday_2  0.001328  0.001328"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_fea_target= data_corr['cnt'].drop(['cnt'], axis = 0) #该Series存储了特征与目标变量之间的相关系数\n",
    "dict_corr_fea_target = {'feature':corr_fea_target.index, 'corr':corr_fea_target, 'abs_corr':abs(corr_fea_target)}\n",
    "corr_fea_target_df = pd.DataFrame(dict_corr_fea_target)[['feature','corr','abs_corr']] #特征、相关系数、相关系数绝对值\n",
    "corr_fea_target_df.sort_values(by='abs_corr',ascending = False) #按照相关系数绝对值排序"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "基于特征与目标变量相关系数的值，特征值之间存在高度相关的情况下可以考虑去除与目标变量相关性低的特征，使用OLS模型是可以考虑如下处理：  \n",
    "1、atemp和temp 去除temp  \n",
    "2、season_1、season_2、season_3、season_4 选择去除season_4  \n",
    "3、mnth_1、mnth_2、mnth_3、mnth_4、mnth_5、mnth_6、mnth_7、mnth_8、mnth_9、mnth_10、mnth_11和mnth_12 选择去除mnth_4  \n",
    "4、weathersit_1、weathersit_2、weathersit_3 选择去除weathersit_2   \n",
    "5、weekday_1、weekday_2、weekday_3、weekday_4、weekday_5、weekday_6和weekday_7 选择去除weekday_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from statsmodels.stats.outliers_influence import variance_inflation_factor\n",
    "X_temp = df.drop([\"instant\",\"yr\",\"cnt\",\"temp\",\"season_4\",\"mnth_4\",\"weathersit_2\",\"weekday_2\"], axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 针对X_temp ，VIF的结果\n",
    "vif = pd.DataFrame()\n",
    "vif[\"VIF Factor\"] = [variance_inflation_factor(X_temp.values, i) for i in range(X_temp.shape[1])]\n",
    "vif[\"features\"] = X_temp.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>VIF Factor</th>\n",
       "      <th>features</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7.538117</td>\n",
       "      <td>season_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10.475338</td>\n",
       "      <td>season_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8.634574</td>\n",
       "      <td>season_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5.801013</td>\n",
       "      <td>mnth_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.062110</td>\n",
       "      <td>mnth_2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>3.257144</td>\n",
       "      <td>mnth_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2.090370</td>\n",
       "      <td>mnth_5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2.639362</td>\n",
       "      <td>mnth_6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>5.696284</td>\n",
       "      <td>mnth_7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>5.409051</td>\n",
       "      <td>mnth_8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>4.742909</td>\n",
       "      <td>mnth_9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>6.252959</td>\n",
       "      <td>mnth_10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>6.349198</td>\n",
       "      <td>mnth_11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>5.558565</td>\n",
       "      <td>mnth_12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>1.691434</td>\n",
       "      <td>weathersit_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1.170726</td>\n",
       "      <td>weathersit_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>22.439970</td>\n",
       "      <td>weekday_0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>1.812331</td>\n",
       "      <td>weekday_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>1.722798</td>\n",
       "      <td>weekday_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>1.725605</td>\n",
       "      <td>weekday_4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>1.734606</td>\n",
       "      <td>weekday_5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>22.233523</td>\n",
       "      <td>weekday_6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>5.949008</td>\n",
       "      <td>atemp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>2.088369</td>\n",
       "      <td>hum</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>1.216603</td>\n",
       "      <td>windspeed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>5.857773</td>\n",
       "      <td>holiday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>107.651379</td>\n",
       "      <td>workingday</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    VIF Factor      features\n",
       "0     7.538117      season_1\n",
       "1    10.475338      season_2\n",
       "2     8.634574      season_3\n",
       "3     5.801013        mnth_1\n",
       "4     5.062110        mnth_2\n",
       "5     3.257144        mnth_3\n",
       "6     2.090370        mnth_5\n",
       "7     2.639362        mnth_6\n",
       "8     5.696284        mnth_7\n",
       "9     5.409051        mnth_8\n",
       "10    4.742909        mnth_9\n",
       "11    6.252959       mnth_10\n",
       "12    6.349198       mnth_11\n",
       "13    5.558565       mnth_12\n",
       "14    1.691434  weathersit_1\n",
       "15    1.170726  weathersit_3\n",
       "16   22.439970     weekday_0\n",
       "17    1.812331     weekday_1\n",
       "18    1.722798     weekday_3\n",
       "19    1.725605     weekday_4\n",
       "20    1.734606     weekday_5\n",
       "21   22.233523     weekday_6\n",
       "22    5.949008         atemp\n",
       "23    2.088369           hum\n",
       "24    1.216603     windspeed\n",
       "25    5.857773       holiday\n",
       "26  107.651379    workingday"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从VIF的结果中还是存在多重共线性的问题，这里先不做处理，尝试用逐步回归筛选变量后再看结果。VIF大于10一般就认为存在多重共线性。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['season_1', 'season_2', 'season_3', 'mnth_1', 'mnth_2', 'mnth_3',\n",
       "       'mnth_5', 'mnth_6', 'mnth_7', 'mnth_8', 'mnth_9', 'mnth_10', 'mnth_11',\n",
       "       'mnth_12', 'weathersit_1', 'weathersit_3', 'weekday_0', 'weekday_1',\n",
       "       'weekday_3', 'weekday_4', 'weekday_5', 'weekday_6', 'atemp', 'hum',\n",
       "       'windspeed', 'holiday', 'workingday'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_temp.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "27"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(X_temp.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面是逐步回归的代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_boston\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import statsmodels.api as sm\n",
    "\n",
    "def stepwise_selection(X, y, initial_list=[], threshold_in=0.01, threshold_out = 0.05, verbose = True):\n",
    "    \"\"\"\n",
    "   基于statsmodels.api.ols中的p值执行前后特征选择。\n",
    "   参数：\n",
    "   X - 所有特征变量组成的数据框\n",
    "   y - 目标变量                 \n",
    "   initial_list - 初始特征名列表              \n",
    "   threshold_in - 进入p值的阈值       \n",
    "   threshold_out - 剔除的p值的与阈值              \n",
    "   verbose-是否打印包含和排除的序列             \n",
    "   返回：所选特征的列表              \n",
    "   选择threshold_in<threshold_out，以避免无限循环。      \n",
    "    \"\"\"   \n",
    "    included = list(initial_list)\n",
    "    while True:\n",
    "        changed=False\n",
    "        # forward step\n",
    "        excluded = list(set(X.columns)-set(included))\n",
    "        new_pval = pd.Series(index=excluded)\n",
    "        for new_column in excluded:\n",
    "            model = sm.OLS(y, sm.add_constant(pd.DataFrame(X[included+[new_column]]))).fit()\n",
    "            new_pval[new_column] = model.pvalues[new_column]\n",
    "        best_pval = new_pval.min()\n",
    "        if best_pval < threshold_in:\n",
    "            best_feature = new_pval.argmin()\n",
    "            included.append(best_feature)\n",
    "            changed=True\n",
    "            if verbose:\n",
    "                print('Add  {:30} with p-value {:.6}'.format(best_feature, best_pval))\n",
    "        # backward step\n",
    "        model = sm.OLS(y, sm.add_constant(pd.DataFrame(X[included]))).fit()\n",
    "        # use all coefs except intercept\n",
    "        pvalues = model.pvalues.iloc[1:]\n",
    "        worst_pval = pvalues.max() # null if pvalues is empty\n",
    "        if worst_pval > threshold_out:\n",
    "            changed=True\n",
    "            worst_feature = pvalues.argmax()\n",
    "            included.remove(worst_feature)\n",
    "            if verbose:\n",
    "                print('Drop {:30} with p-value {:.6}'.format(worst_feature, worst_pval))\n",
    "        if not changed:\n",
    "            break\n",
    "    return included"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\hp\\Anaconda3\\lib\\site-packages\\numpy\\core\\fromnumeric.py:2389: FutureWarning: Method .ptp is deprecated and will be removed in a future version. Use numpy.ptp instead.\n",
      "  return ptp(axis=axis, out=out, **kwargs)\n",
      "C:\\Users\\hp\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:42: FutureWarning: \n",
      "The current behaviour of 'Series.argmax' is deprecated, use 'idxmax'\n",
      "instead.\n",
      "The behavior of 'argmax' will be corrected to return the positional\n",
      "maximum in the future. For now, use 'series.values.argmax' or\n",
      "'np.argmax(np.array(values))' to get the position of the maximum\n",
      "row.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Drop mnth_2                         with p-value 0.988838\n",
      "Drop mnth_1                         with p-value 0.947032\n",
      "Drop weekday_4                      with p-value 0.828757\n",
      "Drop mnth_8                         with p-value 0.748915\n",
      "Drop mnth_12                        with p-value 0.673882\n",
      "Drop mnth_11                        with p-value 0.655399\n",
      "Drop weekday_3                      with p-value 0.62119\n",
      "Drop weekday_5                      with p-value 0.62814\n",
      "Drop mnth_6                         with p-value 0.496164\n",
      "Drop weekday_1                      with p-value 0.302681\n",
      "Drop mnth_5                         with p-value 0.13226\n",
      "Drop mnth_3                         with p-value 0.145572\n",
      "Drop holiday                        with p-value 0.0641458\n",
      "Drop weekday_0                      with p-value 0.232843\n",
      "Drop season_2                       with p-value 0.0614828\n",
      "resulting features:\n",
      "['season_1', 'season_3', 'mnth_7', 'mnth_9', 'mnth_10', 'weathersit_1', 'weathersit_3', 'weekday_6', 'atemp', 'hum', 'windspeed', 'workingday']\n"
     ]
    }
   ],
   "source": [
    "result = stepwise_selection(X_temp, df[\"cnt\"],initial_list=list(X_temp.columns),threshold_in=0.03, threshold_out = 0.06)\n",
    "print('resulting features:')\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "逐步回归后的特征列表：['season_1', 'season_3', 'mnth_7', 'mnth_9', 'mnth_10', 'weathersit_1', 'weathersit_3', 'weekday_6', 'atemp', 'hum', 'windspeed', 'workingday']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 针对筛选后特征，VIF的结果\n",
    "vif = pd.DataFrame()\n",
    "vif[\"VIF Factor\"] = [variance_inflation_factor(X_temp[result].values, i) for i in range(X_temp[result].shape[1])]\n",
    "vif[\"features\"] = X_temp[result].columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>VIF Factor</th>\n",
       "      <th>features</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2.116837</td>\n",
       "      <td>season_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3.239777</td>\n",
       "      <td>season_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.719886</td>\n",
       "      <td>mnth_7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.411935</td>\n",
       "      <td>mnth_9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.233501</td>\n",
       "      <td>mnth_10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2.996491</td>\n",
       "      <td>weathersit_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1.160829</td>\n",
       "      <td>weathersit_3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1.756912</td>\n",
       "      <td>weekday_6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>17.888394</td>\n",
       "      <td>atemp</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>14.300333</td>\n",
       "      <td>hum</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>5.131295</td>\n",
       "      <td>windspeed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>4.659401</td>\n",
       "      <td>workingday</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    VIF Factor      features\n",
       "0     2.116837      season_1\n",
       "1     3.239777      season_3\n",
       "2     1.719886        mnth_7\n",
       "3     1.411935        mnth_9\n",
       "4     1.233501       mnth_10\n",
       "5     2.996491  weathersit_1\n",
       "6     1.160829  weathersit_3\n",
       "7     1.756912     weekday_6\n",
       "8    17.888394         atemp\n",
       "9    14.300333           hum\n",
       "10    5.131295     windspeed\n",
       "11    4.659401    workingday"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "经过逐步回归筛选后的特征也有多重共线性的问题，这里作业不是重点，不在这里详细研究"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据准备"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 从原始数据中分离输入特征x和输出y\n",
    "y = df[\"cnt\"]\n",
    "\n",
    "X1 = df.drop([\"cnt\",\"instant\",\"yr\"], axis = 1) #未考虑多重共线性的\n",
    "\n",
    "#特征名称，用于后续显示权重系数对应的特征，在后面再实现\n",
    "feat_names1 = X1.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当数据量比较大时，可用train_test_split从训练集中分出一部分做校验集；\n",
    "样本数目较少时，建议用交叉验证。\n",
    "在线性回归中，留一交叉验证有简便计算方式。\n",
    "\n",
    "下面将训练数据分割成训练集和测试集，只是让大家对模型的训练误差、校验集上的测试误差估计、和测试集上的测试误差做个比较。  \n",
    "按照作业要求对全体数据，随机选择其中80%做训练数据，剩下20%为测试数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(584, 32)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#将数据分割训练数据与测试数据\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# 随机采样20%的数据构建测试样本，其余作为训练样本\n",
    "X1_train, X1_test, y_train, y_test = train_test_split(X1, y, random_state=33, test_size=0.2)\n",
    "X1_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3、确定模型类型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 尝试缺省参数的线性回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>workingday</td>\n",
       "      <td>3.800799e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>holiday</td>\n",
       "      <td>3.800799e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_4</td>\n",
       "      <td>1.917491e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_2</td>\n",
       "      <td>1.917491e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_3</td>\n",
       "      <td>1.917491e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>season_1</td>\n",
       "      <td>1.917491e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>temp</td>\n",
       "      <td>4.984000e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>atemp</td>\n",
       "      <td>5.540000e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-1.706750e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>hum</td>\n",
       "      <td>-2.898500e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>-1.830830e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>-1.830830e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-1.830830e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>-5.589601e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>-5.589601e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns          coef\n",
       "31    workingday  3.800799e+16\n",
       "30       holiday  3.800799e+16\n",
       "3       season_4  1.917491e+16\n",
       "1       season_2  1.917491e+16\n",
       "2       season_3  1.917491e+16\n",
       "0       season_1  1.917491e+16\n",
       "26          temp  4.984000e+03\n",
       "27         atemp  5.540000e+02\n",
       "29     windspeed -1.706750e+03\n",
       "28           hum -2.898500e+03\n",
       "16  weathersit_1 -1.830830e+16\n",
       "17  weathersit_2 -1.830830e+16\n",
       "18  weathersit_3 -1.830830e+16\n",
       "25     weekday_6 -5.589601e+16\n",
       "19     weekday_0 -5.589601e+16\n",
       "12        mnth_9 -6.237891e+16\n",
       "13       mnth_10 -6.237891e+16\n",
       "6         mnth_3 -6.237891e+16\n",
       "11        mnth_8 -6.237891e+16\n",
       "4         mnth_1 -6.237891e+16\n",
       "8         mnth_5 -6.237891e+16\n",
       "7         mnth_4 -6.237891e+16\n",
       "15       mnth_12 -6.237891e+16\n",
       "5         mnth_2 -6.237891e+16\n",
       "9         mnth_6 -6.237891e+16\n",
       "14       mnth_11 -6.237891e+16\n",
       "10        mnth_7 -6.237891e+16\n",
       "22     weekday_3 -9.390400e+16\n",
       "24     weekday_5 -9.390400e+16\n",
       "21     weekday_2 -9.390400e+16\n",
       "23     weekday_4 -9.390400e+16\n",
       "20     weekday_1 -9.390400e+16"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 线性回归\n",
    "#class sklearn.linear_model.LinearRegression(fit_intercept=True, normalize=False, copy_X=True, n_jobs=1)\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "# 1.使用默认配置初始化学习器实例\n",
    "lr = LinearRegression()\n",
    "\n",
    "# 2.用训练数据训练模型参数\n",
    "lr.fit(X1_train, y_train)\n",
    "\n",
    "# 3. 用训练好的模型对测试集进行预测\n",
    "y_test_pred_lr = lr.predict(X1_test)\n",
    "y_train_pred_lr = lr.predict(X1_train)\n",
    "\n",
    "\n",
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names1), \"coef\":list((lr.coef_.T))})\n",
    "fs.sort_values(by=['coef'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.1.1 模型评价"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of LinearRegression on test is 1327.568546214454\n",
      "The RMSE of LinearRegression on train is 1238.6691840853796\n"
     ]
    }
   ],
   "source": [
    "# 使用RMSE评价模型在测试集和训练集上的性能，并输出评估结果\n",
    "\n",
    "#测试集,输出RMSE结果\n",
    "#print ('The r2 score of LinearRegression on test is', r2_score(y_test, y_test_pred_lr)) #r2_score评价\n",
    "print ('The RMSE of LinearRegression on test is', np.sqrt(mean_squared_error(y_test, y_test_pred_lr))) \n",
    "\n",
    "#训练集，输出RMSE结果\n",
    "#print ('The r2 score of LinearRegression on train is', r2_score(y_train, y_train_pred_lr)) #r2_score评价\n",
    "print ('The RMSE of LinearRegression on train is', np.sqrt(mean_squared_error(y_train, y_train_pred_lr))) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeUAAAFsCAYAAADyj6FyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAfi0lEQVR4nO3df5xddX3n8deHMCaUQEJggiExTUBQfk9wiGQVjEYFDRJ8bFt0EXBB4oJ00a5WgVWi1W0tVLes1DatCgjKL6XYWrawKWmEVWhCQwwGTASyJk5DgEIIyo+Ez/5xT8LNZCZz51fmO3dez8fjPubc7/lxv9975857vud8zzmRmUiSpKG3x1BXQJIk1RjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxljXgR8VBEzB7qegyliHh/RPwyIjZHxIzd+LqbI+LgbuZ9OCLuGaDXeTwi3jkQ25IGk6GsptbVH+POf+wz88jMXNzDdqZFREbEnoNU1aF2JXBRZo7NzH/tPLNq+/NViK6PiK9ExKj+vmj1eo/2dztSszCUpQIUEPa/DTzUwzLHZuZY4G3AGcC5g14raYQxlDXi1femI2JmRCyNiE0RsSEivlIttqT6+UzVW5wVEXtExH+PiLUR8UREXBcR4+q2e3Y176mI+Gyn11kQEbdGxPURsQn4cPXaP46IZyKiIyK+FhGvqdteRsSFEbE6Ip6LiD+KiEOqdTZFxM31y3dqY5d1jYjREbEZGAU8GBG/6On9ysw1wL1AW932x0XEN6p6r4+IL27rSUfE6yPinyPi2Yh4MiJu6tSm11fT+0fED6q23A8cUrfcTnsqImJxRHykmj4kIv6peq+fjIgbImJ8N+9Fd5+xNOQMZWlHfw78eWbuSy0Ubq7KT6p+jq92uf4Y+HD1eDtwMDAW+BpARBwB/AVwJjAJGAdM7vRa84BbgfHADcBW4BPAAcAsYA5wYad1TgHeBJwA/CGwsHqN1wFHAR/spl1d1jUzX6x6v1DrCR/S9eqviog3AicCa+qKrwW2AK8HZgDvBj5Szfsj4E5gP2AK8L+62fTVwAvU3q9z6V1PPIA/Bg4CDqf2fizoZtnuPmNpyBnKGgn+tup9PhMRz1ALy+68DLw+Ig7IzM2Z+ZNdLHsm8JXMfDQzNwOXAB+oenO/A/xdZt6TmS8BnwM6X2j+x5n5t5n5Smb+JjOXZeZPMnNLZj4O/BW1XcX1vpyZmzLzIWAlcGf1+s8Cd1ALxN7WtVEPRMTzwCpgMdX7GBEHAu8BPp6Zz2fmE8BXgQ9U671Mbff4QZn5QmbuNHir6lX/R+Bz1TZWUgv6hmTmmsy8q/onYyPwFXZ+77bpzWcs7VaGskaC0zNz/LYHO/c+650HHAY8HBH/EhGn7mLZg4C1dc/XAnsCB1bzfrltRmb+Gniq0/q/rH8SEYdFxN9HxL9Vu7T/B7Vec70NddO/6eL5WLq2q7o26rhq+2cAbwb2rsp/G2gBOur+8fkrYGI1/w+p9WTvj9pI9656wK1Vferfk7VdLNeliJgYETdWu843Adez83u3TW8+Y2m3MpSlOpm5OjM/SC1QvgzcGhF7s3MvF+BX1AJpm6nUduFuADqo7aoFICL2Avbv/HKdnn8deBg4tNq1eim1MBsIu6prw7LmZuDH1Hr/UAvSF4ED6v752Tczj6zW+bfMPD8zDwI+CvzFtuPIdTZW9Xldpzpu83z187fqyl5bN/3H1N7PY6r37kN0897t4jOWhpyhLNWJiA9FRGtmvgI8UxVvpRYar1A7HrvNd4FPRMT0iBhLrWd7U2ZuoXas+H0R8R+qwVefp+eA3QfYBGyujtteMGAN23Vd++JPgPkR8drM7KB2zPjPImLfalDZIRHxNoCI+N2I2PYPyr9TC8+t9RvLzK3A94EFEfFb1TH5c+rmbwTWAx+KiFFVb7v++Pc+wGZqA/EmA5/qruK7+IylIWcoSzs6BXioGpH858AHquOgvwa+BNxb7aI9Afgm8G1qI7MfozZI6fcBqmO+vw/cSK3X/BzwBLUeZXc+Cfynatm/Bm7axbK91W1d+yIzfwr8M6+G39nAa4CfUQveW6kN2AI4Hrivek9/AFycmY91sdmLqO0e/zfgGuBbneafX73eU8CRwP+tm/d5arvXnwV+SC3gu9PlZ7zrFku7R2R2tVdO0kCqeqfPUNs13VUgSZI9ZWmwRMT7ql2xe1O7YtZPgceHtlaSSmYoS4NnHrUBVr8CDqW2m9RdU5K65e5rSZIKYU9ZkqRC7NaL4B9wwAE5bdq03fmSkiQVZ9myZU9mZmvn8t0aytOmTWPp0qW78yUlSSpORHR5xTp3X0uSVAhDWZKkQhjKkiQVYrceU5Ykde3ll19m3bp1vPCCV/xsJmPGjGHKlCm0tLQ0tLyhLEkFWLduHfvssw/Tpk0jYqBuDqahlJk89dRTrFu3junTpze0jruvJakAL7zwAvvvv7+B3EQigv33379Xez8MZUkqhIHcfHr7mRrKkiQVwmPKklSgBQt2//ZGjRrF0UcfzZYtW5g+fTrf/va3GT9+fK9f6yMf+Qh/8Ad/wBFHHLFD+TXXXMPSpUv52te+1uttAowdO5bNmzc3tOzs2bO58soraW9v3162dOlSrrvuOq666qo+vf7uYE9ZkgTAXnvtxfLly1m5ciUTJkzg6quv7tN2/uZv/manQC5Be3v7oAfy1q1b+7W+oSxJ2smsWbNYv3799udXXHEFxx9/PMcccwyXX345AM8//zxz587l2GOP5aijjuKmm24Car3UbZdU/ta3vsVhhx3G2972Nu69997t2/vwhz/Mrbfeuv352LFjAdi8eTNz5szhuOOO4+ijj+b222/fqW4dHR2cdNJJtLW1cdRRR/GjH/2ooTYtXryYU089FYAFCxZw7rnnMnv2bA4++OAdwvr6669n5syZtLW18dGPfnR70F5wwQW0t7dz5JFHbn8PoHYJ6S984Qu89a1v5ZZbbmmoLt1x97UkaQdbt25l0aJFnHfeeQDceeedrF69mvvvv5/M5LTTTmPJkiVs3LiRgw46iB/+8IcAPPvssztsp6Ojg8svv5xly5Yxbtw43v72tzNjxoxdvvaYMWO47bbb2HfffXnyySc54YQTOO2003YYMPWd73yHk08+mcsuu4ytW7fy61//uk/tfPjhh7n77rt57rnneMMb3sAFF1zAmjVruOmmm7j33ntpaWnhwgsv5IYbbuDss8/mS1/6EhMmTGDr1q3MmTOHFStWcMwxx2yv9z333NOnetQzlCVJAPzmN7+hra2Nxx9/nDe96U28613vAmqhfOedd24P1M2bN7N69WpOPPFEPvnJT/LpT3+aU089lRNPPHGH7d13333Mnj2b1tbazZDOOOMMfv7zn++yDpnJpZdeypIlS9hjjz1Yv349GzZs4LWvfe32ZY4//njOPfdcXn75ZU4//XTa2tr61N65c+cyevRoRo8ezcSJE9mwYQOLFi1i2bJlHH/88dvfk4kTJwJw8803s3DhQrZs2UJHRwc/+9nPtofyGWec0ac6dObua0kS8Oox5bVr1/LSSy9tP6acmVxyySUsX76c5cuXs2bNGs477zwOO+wwli1bxtFHH80ll1zCF77whZ222d0pQXvuuSevvPLK9u2/9NJLANxwww1s3LiRZcuWsXz5cg488MCdzvM96aSTWLJkCZMnT+ass87iuuuu61N7R48evX161KhRbNmyhczknHPO2d7WRx55hAULFvDYY49x5ZVXsmjRIlasWMHcuXN3qNfee+/dpzp0Zk9ZGgKNjqwd6BG4UiPGjRvHVVddxbx587jgggs4+eST+exnP8uZZ57J2LFjWb9+PS0tLWzZsoUJEybwoQ99iLFjx3LNNdfssJ03v/nNXHzxxTz11FPsu+++3HLLLRx77LFA7TjssmXL+L3f+z1uv/12Xn75ZaC2C3zixIm0tLRw9913s3btznc4XLt2LZMnT+b888/n+eef54EHHuDss88ekLbPmTOHefPm8YlPfIKJEyfy9NNP89xzz7Fp0yb23ntvxo0bx4YNG7jjjjuYPXv2gLxmPUNZkgo01P+QzZgxg2OPPZYbb7yRs846i1WrVjFr1iygNijr+uuvZ82aNXzqU59ijz32oKWlha9//es7bGPSpEksWLCAWbNmMWnSJI477rjtg6bOP/985s2bx8yZM5kzZ872nuaZZ57J+973Ptrb22lra+ONb3zjTnVbvHgxV1xxBS0tLYwdO7bbnvLcuXO3X3N61qxZfOxjH+ux3UcccQRf/OIXefe7380rr7xCS0sLV199NSeccAIzZszgyCOP5OCDD+Ytb3lL429mL0RmDsqGu9Le3p7bRuRJI5k9ZXW2atUqDj/88KGuhgZBV59tRCzLzPbOy3pMWZKkQhjKkiQVwlCWpELszsOJ2j16+5kaypJUgDFjxvDUU08ZzE1k2/2Ux4wZ0/A6jr6WetCbwVYOzFJfTZkyhXXr1rFx48ahrooG0JgxY5gyZUrDyxvKklSAlpYWpk+fPtTV0BBz97UkSYUwlCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEJ4SpQ0gDxPWVJ/2FOWJKkQhrIkSYUwlCVJKoShLElSIXoM5YgYExH3R8SDEfFQRHy+Kr8mIh6LiOXVo23wqytJUvNqZPT1i8A7MnNzRLQA90TEHdW8T2XmrYNXPUmSRo4eQzlrN/fcXD1tqR7e8FOSpAHW0DHliBgVEcuBJ4C7MvO+ataXImJFRHw1IkZ3s+78iFgaEUu9T6gkSd1rKJQzc2tmtgFTgJkRcRRwCfBG4HhgAvDpbtZdmJntmdne2to6QNWWJKn59Gr0dWY+AywGTsnMjqx5EfgWMHMQ6idJ0ojRyOjr1ogYX03vBbwTeDgiJlVlAZwOrBzMikqS1OwaGX09Cbg2IkZRC/GbM/PvI+KfIqIVCGA58F8GsZ6SJDW9RkZfrwBmdFH+jkGpkSRJI5RX9JIkqRCGsiRJhfB+ylLBGr0/83C5j3OztUcaaPaUJUkqhKEsSVIhDGVJkgphKEuSVAhDWZKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEIYypIkFcJQliSpEN5PWSOW9+yVVBp7ypIkFcJQliSpEIayJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUCENZkqRCGMqSJBXCUJYkqRCGsiRJhTCUJUkqRI+hHBFjIuL+iHgwIh6KiM9X5dMj4r6IWB0RN0XEawa/upIkNa9GesovAu/IzGOBNuCUiDgB+DLw1cw8FPh34LzBq6YkSc2vx1DOms3V05bqkcA7gFur8muB0welhpIkjRAN3U85IkYBy4DXA1cDvwCeycwt1SLrgMndrDsfmA8wderU/tZXkrYbjHtie59tDaWGBnpl5tbMbAOmADOBw7tarJt1F2Zme2a2t7a29r2mkiQ1uV6Nvs7MZ4DFwAnA+IjY1tOeAvxqYKsmSdLI0sjo69aIGF9N7wW8E1gF3A38TrXYOcDtg1VJSZJGgkaOKU8Crq2OK+8B3JyZfx8RPwNujIgvAv8KfGMQ6ylJUtPrMZQzcwUwo4vyR6kdX5YkSQPAK3pJklQIQ1mSpEI0dJ6ypLL15tza4XAe7nCoozQY7ClLklQIQ1mSpEIYypIkFcJQliSpEIayJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUCENZkqRCGMqSJBXCUJYkqRCGsiRJhTCUJUkqhKEsSVIhvJ+ymo734tXu0Ojvmb+P6g17ypIkFcJQliSpEIayJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUCM9TlqQ6nlesoWRPWZKkQhjKkiQVwlCWJKkQhrIkSYXoMZQj4nURcXdErIqIhyLi4qp8QUSsj4jl1eO9g19dSZKaVyOjr7cA/y0zH4iIfYBlEXFXNe+rmXnl4FVPkqSRo8dQzswOoKOafi4iVgGTB7tikiSNNL06phwR04AZwH1V0UURsSIivhkR+3WzzvyIWBoRSzdu3NivykqS1MwaDuWIGAt8D/h4Zm4Cvg4cArRR60n/WVfrZebCzGzPzPbW1tYBqLIkSc2poVCOiBZqgXxDZn4fIDM3ZObWzHwF+Gtg5uBVU5Kk5tfI6OsAvgGsysyv1JVPqlvs/cDKga+eJEkjRyOjr98CnAX8NCKWV2WXAh+MiDYggceBjw5KDSVJGiEaGX19DxBdzPqHga+OJEkjl1f0kiSpEIayJEmF8H7KkjSIGr0/s/dxFthTliSpGIayJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUCENZkqRCGMqSJBXCUJYkqRCGsiRJhTCUJUkqhKEsSVIhDGVJkgphKEuSVAhDWZKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEIYypIkFcJQliSpEIayJEmFMJQlSSqEoSxJUiF6DOWIeF1E3B0RqyLioYi4uCqfEBF3RcTq6ud+g19dSZKaVyM95S3Af8vMw4ETgI9FxBHAZ4BFmXkosKh6LkmS+qjHUM7Mjsx8oJp+DlgFTAbmAddWi10LnD5YlZQkaSTYszcLR8Q0YAZwH3BgZnZALbgjYmI368wH5gNMnTq1P3XVCLdgwVDXQJIGV8MDvSJiLPA94OOZuanR9TJzYWa2Z2Z7a2trX+ooSdKI0FAoR0QLtUC+ITO/XxVviIhJ1fxJwBODU0VJkkaGRkZfB/ANYFVmfqVu1g+Ac6rpc4DbB756kiSNHI0cU34LcBbw04hYXpVdCvwJcHNEnAf8P+B3B6eKkiSNDD2GcmbeA0Q3s+cMbHUkSRq5vKKXJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUCENZkqRCGMqSJBXCUJYkqRCGsiRJhTCUJUkqhKEsSVIhDGVJkgphKEuSVAhDWZKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKsSeQ10BSbvXggUDu5ykgWNPWZKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIK4XnKklSA3pwX7jnkzcuesiRJhTCUJUkqhKEsSVIhegzliPhmRDwRESvryhZExPqIWF493ju41ZQkqfk10lO+Bjili/KvZmZb9fiHga2WJEkjT4+hnJlLgKd3Q10kSRrR+nNM+aKIWFHt3t6vu4UiYn5ELI2IpRs3buzHy0mS1Nz6GspfBw4B2oAO4M+6WzAzF2Zme2a2t7a29vHlJElqfn0K5czckJlbM/MV4K+BmQNbLUmSRp4+hXJETKp7+n5gZXfLSpKkxvR4mc2I+C4wGzggItYBlwOzI6INSOBx4KODWEdJkkaEHkM5Mz/YRfE3BqEukiSNaF7RS5KkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEIYypIkFaLHy2xKg2nBgqGugbrT6GfjZygNHHvKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEIYypIkFcJQliSpEIayJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUCO+nLKlfvJ+yNHDsKUuSVAhDWZKkQhjKkiQVwlCWJKkQPYZyRHwzIp6IiJV1ZRMi4q6IWF393G9wqylJUvNrpKd8DXBKp7LPAIsy81BgUfVckiT1Q4+hnJlLgKc7Fc8Drq2mrwVOH+B6SZI04vT1POUDM7MDIDM7ImJidwtGxHxgPsDUqVP7+HKSpN5q9BxyzzUvx6AP9MrMhZnZnpntra2tg/1ykiQNW30N5Q0RMQmg+vnEwFVJkqSRqa+h/APgnGr6HOD2gamOJEkjVyOnRH0X+DHwhohYFxHnAX8CvCsiVgPvqp5LkqR+6HGgV2Z+sJtZcwa4LpIkjWhe0UuSpEIYypIkFcJQliSpEIayJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUCENZkqRCGMqSJBWir/dT1gjlfVclafDYU5YkqRCGsiRJhTCUJUkqhKEsSVIhDGVJkgphKEuSVAhDWZKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEJ4P2VJGma8r3nzsqcsSVIhDGVJkgphKEuSVAhDWZKkQvRroFdEPA48B2wFtmRm+0BUSpKkkWggRl+/PTOfHIDtSJI0orn7WpKkQvS3p5zAnRGRwF9l5sLOC0TEfGA+wNSpU/v5cpKkgdboec+eHz34+ttTfktmHge8B/hYRJzUeYHMXJiZ7ZnZ3tra2s+XkySpefUrlDPzV9XPJ4DbgJkDUSlJkkaiPodyROwdEftsmwbeDawcqIpJkjTS9OeY8oHAbRGxbTvfycz/PSC1kiRpBOpzKGfmo8CxA1gXSZJGNE+JkiSpEIayJEmF8H7KTaw35xR6/qGknvg3ZfDZU5YkqRCGsiRJhTCUJUkqhKEsSVIhDGVJkgphKEuSVAhDWZKkQniech812/l6w6GOkoYP79HcN/aUJUkqhKEsSVIhDGVJkgphKEuSVAhDWZKkQhjKkiQVwlCWJKkQnqdcEM/Xk6SRzZ6yJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUCENZkqRCGMqSJBViWJ+n3Gz3NJYkdW0w7s9c4j2f7SlLklQIQ1mSpEIYypIkFcJQliSpEP0K5Yg4JSIeiYg1EfGZgaqUJEkjUZ9DOSJGAVcD7wGOAD4YEUcMVMUkSRpp+tNTngmsycxHM/Ml4EZg3sBUS5KkkScys28rRvwOcEpmfqR6fhbw5sy8qNNy84H51dM3AI/04mUOAJ7sUwXL18xtg+ZuXzO3DWzfcNbMbYPmat9vZ2Zr58L+XDwkuijbKeEzcyGwsE8vELE0M9v7sm7pmrlt0Nzta+a2ge0bzpq5bdD87YP+7b5eB7yu7vkU4Ff9q44kSSNXf0L5X4BDI2J6RLwG+ADwg4GpliRJI0+fd19n5paIuAj4R2AU8M3MfGjAalbTp93ew0Qztw2au33N3DawfcNZM7cNmr99fR/oJUmSBpZX9JIkqRCGsiRJhRjyUI6IT0ZERsQB1fOIiKuqS3euiIjj6pY9JyJWV49z6srfFBE/rda5KiK6Ol1rt4qIP6rqvzwi7oyIg6ryYd++iLgiIh6u6n9bRIyvm3dJVc9HIuLkuvIuL8laDRS8r2rzTdWgwSEVEb8bEQ9FxCsR0d5p3rBvX3eG62VzI+KbEfFERKysK5sQEXdV7/tdEbFfVd7r799QiojXRcTdEbGq+p28uCpvlvaNiYj7I+LBqn2fr8q7/N5ExOjq+Zpq/rS6bXX53Rx2MnPIHtROqfpHYC1wQFX2XuAOaudBnwDcV5VPAB6tfu5XTe9XzbsfmFWtcwfwnqFsV1Wnfeum/yvwl83SPuDdwJ7V9JeBL1fTRwAPAqOB6cAvqA0CHFVNHwy8plrmiGqdm4EPVNN/CVxQwGd3OLUL3SwG2uvKm6J93bS52zaU/gBOAo4DVtaV/SnwmWr6M3W/o73+/g1x2yYBx1XT+wA/r34Pm6V9AYytpluA+6p6d/m9AS7k1b+lHwBuqqa7/G4Odfv68hjqnvJXgT9kx4uOzAOuy5qfAOMjYhJwMnBXZj6dmf8O3AWcUs3bNzN/nLVP5zrg9N3bjJ1l5qa6p3vzahuHffsy887M3FI9/Qm1c9Sh1rYbM/PFzHwMWEPtcqxdXpK16vG/A7i1Wv9ayvjsVmVmV1eea4r2dWPYXjY3M5cAT3cqnkft/YYd3/deff8Gv/a7lpkdmflANf0csAqYTPO0LzNzc/W0pXok3X9v6tt9KzCn+p51990cdoYslCPiNGB9Zj7YadZk4Jd1z9dVZbsqX9dF+ZCLiC9FxC+BM4HPVcVN077KudT+M4fet21/4Jm6gC+tbZ01c/u6a8NwdWBmdkAt2ICJVXlvP8NiVLtqZ1DrTTZN+yJiVEQsB56g9s/CL+j+e7O9HdX8Z6l9z4ptX2/15zKbPYqI/wO8totZlwGXUtsNutNqXZRlH8oH3a7al5m3Z+ZlwGURcQlwEXA5w6R9PbWtWuYyYAtww7bVulg+6fqfv6I/u+5W66KsyPb1wXCqa38U9T1rVESMBb4HfDwzN0X3w0qGXfsycyvQVo1NuY3a4aOdFqt+Drv29daghnJmvrOr8og4mtp+/werX64pwAMRMZPuL9+5DpjdqXxxVT6li+UHXXft68J3gB9SC+Vh0b6e2lYNFDkVmFPtVoddX3q1q/Inqe1e27P6r7fEz67esGlfHzTbZXM3RMSkzOyodt8+UZX39vs35CKihVog35CZ36+Km6Z922TmMxGxmNox5e6+N9vaty4i9gTGUTt00Ty/v0N9ULv6e/44rw70msuOAxXuz1cHKjxGbZDCftX0hGrev1TLbhsI9d4C2nRo3fTvA7c2S/uoHYv6GdDaqfxIdhxs8Si1AUR7VtPTeXUQ0ZHVOrew44COC4f6s6trz2J2HOjVVO3r1NZu2zAcHsA0dhzodQU7DoT602q619+/IW5XUBtH8j87lTdL+1qB8dX0XsCPqP2z3+X3BvgYOw70urma7vK7OdTt69N7MtQVqN7Qx3k1lAO4mtpxhZ92+qN4LrUD+GuA/1xX3g6srNb5GtWVyoa4Td+r6rQC+DtgcrO0r6rfL4Hl1eMv6+ZdVtXzEepGiVMbFfrzat5ldeUHUxtdvqb6Io4u4LN7P7X/vF8ENgD/2Ezt20W7u2xD6Q/gu0AH8HL1uZ1H7TjjImB19XPbP7i9/v4NcdveSm037Iq679t7m6h9xwD/WrVvJfC5qrzL7w0wpnq+ppp/cN22uvxuDreHl9mUJKkQQ31KlCRJqhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIK8f8BRMrfhTRewIUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#在训练集上观察预测残差的分布，看是否符合模型假设：噪声为0均值的高斯噪声\n",
    "f, ax = plt.subplots(figsize=(7, 5)) \n",
    "f.tight_layout() \n",
    "ax.hist(y_train - y_train_pred_lr, bins=40, label='Residuals Linear', color='b', alpha=.5); \n",
    "ax.set_title(\"Histogram of Residuals\") \n",
    "ax.legend(loc='best');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "残差分布和高斯分布比较匹配，但还是大值这边的尾巴更长"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "多元线性回归的残差分析，有多个残差图，一般来说，残差不正态可能有一下原因：  \n",
    "1、数据中有异常值  \n",
    "2、方程有弯曲  \n",
    "3、方程有失拟项  \n",
    "4、Y值残差需要转换后才能正态  \n",
    "解决方案：  \n",
    "1、剔除出异常值（如果能找到异常的原因）  \n",
    "2、增加方程的高次项  \n",
    "3、因为没有找全  \n",
    "4、Y残差进行正态性转换"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAADQCAYAAADcQn7hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO2df5hcdXnoP+/OTpLZoEyiwSZLIJHmhgtispgKmHtbwJaI/NoiCjx6i7aV/qC3Qn22Li1PkygtqbGKtn1UbpFLK8UA0jUQNSDg7VN8iCRufhghJfIr2VCNTTbaZEgmu+/945wzOTN7zpkzM+fMj9338zzJ7pw5c847Z+e8877v9/0hqophGEYadLVaAMMwJi+mYAzDSA1TMIZhpIYpGMMwUsMUjGEYqdHdagHS4M1vfrMuWLCg1WIYxpRhy5YtP1PVOZXbJ6WCWbBgAZs3b261GIYxZRCRV4K2m4tkGEZqmIIxDCM1TMEYhpEakzIGYxhTgaHhEdZu3MW+0QLz8jkGViymv6+37v3SwBSMYXQgQ8Mj3PrwDgrFMQBGRgvc+vAOgDLlEXe/tDAXyUidoeERlq95koWDG1i+5kmGhkdaLVLHs2r9zpLS8CgUx1i7cVfZtrUbd8XaLy3MgjFSpdXfoJWytMpVSJKh4RFGC8XA5/aNFiIfV9ueNGbBGKnS6m9QD0/RjYwWUE4ouk60pqKu3bx8LvJxte1JYwrGSJVavkHTdKXaRdElQZT1MbBi8YTHuWymbFsum5mwX1qYgjFSJe43aNoWRqtdhSQJu6azerITXL7+vl7uuPocevM5BOjN57jj6nNsFcmYHAysWFwWg4Hgb9AoC8O7GRqJoczL5xgJUCbNchWSJOyarrzi7MD9+/t6WxZrksnYMnPZsmVqtUjtQxzFsHBwA2GfRAFOzmU5fOw4xTEt264438r+YwadDwi8KZv5bd4IQ8MjrH5kJwePOMHdXLaLGdkMo0eKbRGwFpEtqrpswnZTMEY7sHzNk4EWRlw8ZQHhigSo2wJq1QrU0PAIq9bvDFw1ynYJa9+/pC0UpCkYo62pXM6uh4wIb8x1l77l/fTmczw9eHFisjXD+olzTRp5X5XnakSBhikYi8EYbYH3YfY+5PV87Y2pBioXqC2YW3mzHTl2PDA+9PEHtpXJniRDwyN8/IFtjFUxAJIIUqeZq2QKxmgb/MHIvk8+Fqos6iFuMDfoZgtjTDWRG7FSoV105hy+vmWkqnKB8vdVrxUSJ8BeL6ZgjKq0Iv6QpOdeS95H0M0WRa03Ypgy8Su0+555NZYFl+2S0vtqxApJcwnf8mCMSILyU25et5Wlqx+rK0clbjLdoZBU+Djkc9m68z7quaniviboWt73zKsTFFpc3eoP8DaSSJhmtq9ZMEYkYd/oo4Vize5BLd+yYXkrcbh8yVxu7z+nrteGnTefy/KL148Hui1xb8Sga1mvodabz9Hf11uyiMKuVRzlFzdXqR7MgjEiifqA1ppqX8u37MCKxWS7JL6gPp56fn9drxsaHuHIseMTtueyGVZdeTZ/84ElDaXdJ5U17J3ztqEd3LJua6QijqP80sz2NQvGiKSaJRF20wTFbWrx9fv7essSy2qhnhs5bEk4n8uy6sqzy262pLOJvYTBOPT6EgfjxGqOHDvO0PBIVRnTyvY1C8aIJKhYzk/QN2RYXVG+Jxt4DIXAeMxonatI3vFuG9pB3ycfY8HgBhYMboiMG4W5gjOnd5fdeP19vQysWMy8fI59owXWbtwVOxYVVnj4wfNPK7Meqh1j7cZd3LxuayyldPBIsaVV42bBTDKSXvHxXhtkTYS5B2Gu0PTuLnLZTOCNHBSPyfdk616qHhkt8NVnXi3bNlooMvBgcO5KXOuqlk5yQeUKM7JdpdcGWUcAS1c/Fpi525PtqisZMe2cnSjMgplEpFWR3N/Xy/BfXMKd1y6N5aeH3ayHCsWSrx9EoTjGqvU7S6tMYU2VGqE4roExn7grKXHiSEF/h4GHtjHw4LYyhXn0+PiE8w0NjyABoacuYFp3sHKOg5ez02xLxiyYSUSaCVMQ30+Pqlz2jhFW3DhaKJ5QLClVsfgVoH8VpjIWEmShxbF0gv4O/iJND0+h+os0wyyUTEYaVrhJfhbiYgpmEtEuPU/iLHs2sgzdKAosGNyAiBNgHdcT2z0qK7Q94rR9qOV6jxaKpSBsVJJfkIKqh2Z/FlJ1kUQkLyIPicjzIvKciFwgIrNF5HERecH9OcvdV0TkCyKyW0S2i8i5vuPc4O7/gojckKbMnUyr2yN6xFn2rBY8bgaqJ5SLH4HQ2FWcDnG1Xm/PvWrGzV9LyUQS3QXTtmA+D3xbVa8RkWlAD/BnwBOqukZEBoFB4BPApcAi9995wBeB80RkNrASWIbzJbNFRNar6sGUZe840kyYCiMsqFzNnaosbpyXzzF65BiHj9VfTZ0U6soVJH9Q0Ht694nv6aHhEQ4fnZhLE8XIaIHla55sKKhdST6X5ejx8QkW0eGj1Zetkyx+TE3BiMgbgV8FPgygqseAYyJyFXChu9u9wHdxFMxVwD+q0z/iGdf6mevu+7iqHnCP+zjwHuD+tGTvVIJu2jTrhoaGRxh4aFvJfPeCmX5Zqsnr32/p6seA1isYqG5NvF48EaD1spo3v3KgrK6oFpJ0F8WVKZ/L0iWUKe04GdhJxvLStGDeCuwH7hGRJcAW4GPAW1T1NQBVfU1ETnH37wX2+F6/190Wtr0MEbkRuBHgtNNOS/addBDNbI+4+pGdE2IDxTFl9SM7Y8lQaf2ksWpUL2E5OxB+A96/aU+sCug08QeqRwtFgnKhqymLJGN5acZguoFzgS+qah9wGMcdCiPoWmjE9vINqnep6jJVXTZnzpx65DVqJMycj2PmBy3l1lcYkA5hemJoeCTU2mi1cunN5ybcGGESeW5ZUIwlyVhemgpmL7BXVTe5jx/CUTg/cV0f3J8/9e0/3/f6U4F9EduNDibJwr80GC0UuW1oR9k2zyVsRzIiNVsYYflSSY46SU3BqOp/AHtExJPq3cCPgPWAtxJ0A/AN9/f1wG+5q0nnA4dcV2ojcImIzHJXnC5xtxktJp8LdiPCtvtph3Eh1VLzv/rMq2VKJsglbBeuP29+Q6uF/mTBJIsf015F+t/Afe4K0ovAR3CU2gMi8jvAq8D73X2/CbwX2A0ccfdFVQ+IyKeAZ939PukFfI3WsurKsxl4cBtF31pvtktYdWXw+Aw/rcyD8csQtPLm5/5Ne0qtH5LssJckPdkubu8/JzBRr5ZCSv/fI6lYXqoKRlW34iwvV/LugH0VuCnkOF8BvpKsdEajVFu1iqqLGlixmJvXbW2Z7J7J78kTJkur4yrVyGUz/NXV55SudaE4RkaEMVV63Y55cTvkZYJqFBrEMnmnOI0WR4Z90wXlUtyybiubXznAstNnN31k6/IzZvPyfxYC32d/X29og+00brokmZHtmrA8PqZapkAriz7DSEOZWrHjFCbNca1hQdyvPvMqAw9ta6p79KHzT+O+j14Q2Wbh+vPmB77Wvz0stpTLdpXiFc1WRwePFAPbbvpjKtVaQHjE3a8WzIKZwqRZHBkVxG12oPTRba8BTGiu7U848+IsXi5LRoTrz5tf1nozLOZ0x9VvL12vBYMbmvKe/IRdzX2jhdAufZWklfFtg9emMGEVzQK8tOayqq+Pcq8andSYNGHBzriDy/xV15Uxjqee31+6Bu30nmf1ZHm9OLFcIJ/LcvmSuWVyN5rxbYPXjAk0MhA+rF5l8ysHeOr5/W11o0H0t3wUtw3tmBAk9WIcQSNHwugSp7CyllWdRjlaHKNQnNhzZub07rqboteKxWCmMI0kVIW5V/c982rTlMvMaY1XY5+cy4ZmtN42tIOvhqzAeKUBceqOshnhsx9YGphpmwRhcZ8jAcoFmpuDZBbMFKaR4siwD2nQDZQRYVw1cRei0crrbJdw+NjxUg2UN/Mp7vJ5nFUXAa79lfml4yfNrDoqsJvZvsMUzBSn3oSqWpTFuGopptMusZled+Z02slzCvzLD0ZY9+yeqvvWw+vF8VAlE9SyIe32HZWYi2REEtZ4qJaGUSf7lneb8eHOZTN86PzwinoBnh68uO6pBbVy+NhY7JWzWptwFYpjqE58nTfLKa15R3ExC8YIJU7joTgNo/y5av19vaxavzN2a4ZpGeFYjcvaheIYG7a/Rj6XDTyP5yIk7bJ5q0uNvP597+iNnXnrcahQ5HPXLg11dZs9ScBPVQUjIm8B/gqYp6qXishZwAWqenfq0hktIWocqTcC45Z1Wyd8kBeG5IBUWgqXL5kbK7u0C2pWLh4HjxTpEifO4s9b8bsIAysWc0vM+ULV8JLwGulpM6bKU8/vr1kefzP1diOOBfN/gXuAP3cf/zuwDjAFMwmJ6mzv4X1LV1o0YRaB12Tba6Qdd7Rr8BpIfMbVybI9pWfahPlEy9c8yb7RAj3TMpHB4mxGyHZJ6IqMR1LNsqIsqqBYS7NjKrUSJwbzZlV9APfvrarHaZe+hkbiRHW2D8Kfkn7RmdGNvrx6pGYGeQ8fG+PpwYt5ac1lpYQ6f3nE4WNjZDNCTzb4ViiOKX2n5RsqAch2CYtOmdnAEZz4SS2zqdqFOBbMYRF5E+4KpNerJVWpjNQJy8KtJ0fCe00cy6TVeeNhM4tOmt7NrJnTA5Xf9358gHedMZvv/fhAffIL3HTRIv7s4e1VLaEg/FZKu7pCYVQtFXDHh/wt8Dbgh8Ac4BpV3Z6+ePVhpQLRBLlBuWyGO64+JzT2AuFBzHwuy8zp3S1bfq6WHSuc6P2SVMylVmb1ZFl5xdl1jX6989qlba9UwkoFqrpIqvoD4NeAdwG/B5zdzsrFiGZoeISPP7AttMgxLLv3zmuX8jcfWBK4jDpaKLY0t6WawvBXivdEZP+m2ZrBi51ULhtX6/7X6wZwO5WqCkZEbgJOUtWdqvpD4CQR+cP0RTPqJSx3xbNcwpZS940WytolgnPT+eMsUbOl251CcSwyoJt2cymvSv3pwYv53LVLgfDO/9D+Adw4xAnyflRVR70H7sCzj6YnktEIUT1eqgVw5+VzE2Y1B60YDaxY3FYTADoFL1bl/xtB+egMz4rqhABuHOLEYLYDS9yWlohIBtiuqtUbr7aIqRyDCUvF73UbLYX9tXPZDO97R2/VwWGNJpN1Cmm8z2qxqritI9qRumMwOB38HxCRd4vIxTgTFb+dtIBGMkQNzQorchNxxp9+NaAzWiVJ33RdEr8LnH9Ea9qMq8aajhAXr7AyKlbVDpMWkibOX+wTwJPAH+A05X4C+NM0hTLqJ2poVlAAN9sldHdJy6Yqjmv8pesuEWZFTFxMknn5HIcSuia9+RwnzeiuWo/UzCrnZhFnFWlcVb+oqteo6vtU9cuqaol2bUpUj5egAG5xXNt21k8lYYV9SSM4MaeuBFaVPLenWmHlZAjoBhGqYETkAffnDhHZXvmveSIatVBtaFZ/X29JCXViLOVQoVj2/tIINntXpdHr41caUdbJZAnoBhEa5BWRue5w+tODnlfVV1KVrAGmcpA3Du3Sk6Ue/IHQoeGRpsxWqiXg62+uVTkjKiy5cTIolpp78rrKJQPcraq/nqp0RqJUm3VUTzCxmb1kw6h0I5o1WymucslmhLXXLAlUGI10D6xGo7Ot0iSyFklVx0TkiIicrKpWf9QBxOnhEtUHZea0DMeOj09ocfC+d/SWutC3StHMqChIbLdVl5nTuiNv7DTqiOL8vVtJnFWk14EdInK3iHzB+5e2YEZ9RM068ohKlMv3TGPt+5eUYhz5XJYZWWcJ+z8Ovd5SK+bgkSIDD20rZSa326pLUqtOtRDn791K4lRTb3D/GW1ErdXQI+4QLnA+lFFjPLxv2sqRHe0QFC6OKasf2VkKVtdTPFiN3jo73bVC4UXlPbUDVRWMqt4rItOAM3Hc8F2qeix1yaYI9fjPUWZxlPsz8NA2UMrcn0q8m2RoeKTm1o3Nwisc7O/rZfMrBxKVUyByYLzXNCsoYNuKZeZGZls1gzjFju8Ffgx8Afg7YLeIXJq2YFOBemdDR5nFUc24i2MaqVz8N0mUldMOeMWc92/ak6icCqHHFE40LfdnFc/qybZsNaiR2VbNIE4M5rPARap6oar+GnAR8Lm4JxCRjIgMi8ij7uOFIrJJRF4QkXWudYSITHcf73afX+A7xq3u9l0isqKWN9jO1Os/R7lB4FQ810plLkazTexFp8ysqV2Cp5TTcNvCjultvfXhHWWZz6/X0UQqKarlPbWaODGYn6rqbt/jF4Gf1nCOjwHPAW90H/818DlV/ZqIfAn4HeCL7s+DqvrLInKdu9+1bpPx64CzgXnAd0Tkv3VCNnG9y8XVbu4oN+jWh3eUPnBx4whd4pzTU2xR/XXrpdoy996Drzc9xhOW3xK2vTefi/xSaNVN3c5d7uJYMDtF5Jsi8mERuQF4BHhWRK4WkaujXigipwKXAf/gPhbgYuAhd5d7gX7396vcx7jPv9vd/yrga6p6VFVfAnYD74z9DltEHPcnqm4oiig3qFAcY9X6nRw4fDS2rF49kF/GWuYexUFxFFkQXUKsQG2SxYcCgQ20ctkM1583P9TtaPegarsRR8HMAH6C09XuQmA/MBu4Ari8ymvvxCmM9GzINwGjbuNwgL2Ap3p7gT1Qaix+yN2/tD3gNSVE5EYR2Swim/fvj9e1Pk3iLhfX4z97ZnEYo4Vi4NDzOPi/jZNsLtWbz/HZDywlmynXMtmMEBEWKjFzWoatKy9JTJ58T7b0N/L3YPHyfSq3e25HvV8KU5U4q0gfqefAInI5jnu1RUQu9DYHnaLKc1Gv8ct5F3AXOKUCNQucMHG+6RrN7kyrN8vIaKE01mNePsfMKqM9/GTd/gv+Akp/sSVMfL9RfYBLx80434VJ9dU9eKRYWo0aUyWXzXDRmXPK+uF42/2yD6xYzMCD28qC5dkuaZugaruR5mTH5cCV7irUDJwYzJ1AXkS6XSvlVGCfu/9eYD6wV0S6gZOBA77tHv7XtC1xlw8rbzp/HCSMaq0vG8WrJoZ4A9t7sl0UiuNlc4eipgxWvrfNrxyoOohttFBkaHiE/r7emuuPerJdiEikkiwUx7h/054J1zQwvlL5lWft/UJJrYOPqt6qqqeq6gKcIO2TqvpB4CngGne3G4BvuL+vdx/jPv+k20VvPXCdu8q0EFgEfD8tuZMirvtTz1J1rbOLKsmIlLJ0K12WemqOZs2cXjZ3qFaLLO4gtj95YCt9n3ysRulAkZIFFEVUr2KPtRt3TWhvURzTtsmcbTdaMZv6E8DXROR2YJgTEyLvBv5JRHbjWC7XAajqTrd1xI+A48BNnbCCFNf9CYvVfPyBbWXH8dNIQLGygrdypauelaPKXrNRdTFDwyOsfmRnyT0Jmx8dxLgyYbKhHxHIdXdNmD1UKI7FUshhLqff6rQgb22EKhgR+ZOoF6rqZ+OeRFW/C3zX/f1FAlaBVPV14P0hr/9L4C/jnq9diLN8GPbBHFMNLVqrVxH0Bii5SkVYT1zHuwGrLeEODY8w8NC2MgsgyU56qtQ12AzCexJXWp3tnjnbbkRZMG9wfy4GfgXHVQFn9ehf0xRqKhGlLMLyKy46c06s4fF+/Fmo/gBuUGCzVrzjVvt2D3IvmsWsniz/dfR44Pm9oWj9fb0sO312mYVV2Qe4ncoEOoGofjCrAUTkMeBcVf2F+3gV8GBTpGtDku69Ua1gb59bpOidM9+T5VCV9otBKLD6kZ28Xhwvc2EareOZ1ZOt2gaiS4SFgxtaWnqw8gpnCEale7bqyrMn/P38mbmjhWKZJZlmX5fJSJyxJc/jjC056j6eDmxT1TObIF9dpNXRLq2uZN60xbCxrEePj8eKITS7KZQAHzz/NG7vd3Jygq5PLcfyy57NSFVrJ+77zeeybF15SSw5osa+dOpIkWbQyNiSfwK+LyKrRGQlsAn4x6QF7ATS6r3R39cbmlUqMbNcwbnZlp8xuyFZPOLUBSnw9S0jpRWvyrqYuLVFXTiKyl9Ps/aaJZFJdbN6snzu2qVVE+9y2Qyrrow/wsuCuMkSJ9HuL0XkW8D/dDd9RFWH0xWrPUnzw+dZQKvW7ywFPmdkuyJXTYJ45sWDoc+FfeNXbvcCnnHcp8o4kd+NWDgYr41QJiMsO312yRLyU2kRVVpNYfsowUHtalgQN1niLlP3AD9X1XtEZI6ILHTrgqYUzfjwHT1+wv+vVblAdJA26JlslzCtu6uUhOaPS8QNJIcFqU+OuQTt5ZFUKoI48Y6kYyIWxE2WqgrGdYuW4awm3QNkga/iZOpOKdL+8DWaQAe1lQ+I+58/w9Wv4OIeK8wVqmWsUJgVGGepP8lqYgviJkscC+Y3gT7gBwCquk9E3hD9kslJ2h++JFytad3C8XFiLQcrE/fzuzxxFVXYftWGjfnpEimVArSadm5/0GnEUTDHVFVFRAFEZGbKMrU1cT589S5l53uydblFfgrFcbJdzojV0SNF5uVzHD56vKaENk/Rxe0pExZorSUhMCqx0Ohc4iiYB0TkyzhFih8Ffhu3v4sxkUbGSCRVu+hV+r605rJAmcBx7cKCyF5MKU7lsleF7CXvnZzLIuJYLye7tU5xk+ta3bjJSJ44s6k/g9MA6us4cZi/UFUbWxJCI0vZSY69OHikGLp87PU3WXnF2ZEFmf19vZHKZVZPtpRe7xVrjhacNgje72PjWorFZET40Pmn8fKay0ILkG05eHIRJ8j716r6CeDxgG1GBY0sZSfdpvLjD2zjlnVbS25aWKJYlDsX5SaNHilWXWnyN5MaU+XrW0ZYdvrsyKzfdonFGI0Tx0X6DZwKaD+XBmwzqH8pe2h4hMNHj0fuUyte8NVz0za/cqA0ndGvTKJu5qhShno8ukJxjJvXbS21iqh0nywWM7mIqqb+A+APgTNEZLvvqTcA30tbsE6llqVsLxg8MlpIPc2/UBwrS5yLGxvynkt6yPxooUi2SxCZGHuyWMzkIcqC+WfgW8AdwKBv+y9U9UCqUnUwcZeyKwOvzaghqjxHLTdyGgowakaTxWImB1HV1IeAQyLyeeCAr5r6DSJynqpuapaQnUacpewkkuqSIOxG9i+1d4k0vRLaUvMnB3FiMF8EzvU9PhywzahCEp3j0iDoRq60ruIk3AnheTxBbpCfbJeUWTOWmj95iFNNLerr6aCq47Sm1WbHEtR3t9ksOmVm7BEpqx/ZWbN1pTg9Vyp7/AJ0u8vTYUvTJ83obtvJhEZjxFEUL4rIH+NYLeAEfl9MT6TJRz03bK1kRBhXZV4+x4I35XjmxYNllseRY+O87x29bNj+WmC3Nn/AuR568zn6+3r5s4e3T2yKPa489fz+UDdr9EiR4b+I16/F6CziKJjfxxl8fxvOF9UTwI1pCtXJVLpCF505p+H0/2oENb0Kyihe9/09ZSM2vG5tm185MKEXbS147TiHhkdCe+LuGy2E5tRYvGXyUrWjXSeSVke7agSl5Ne7+lItbuER1vMkrDNbELVWYGvF43edMZuX/7MQeT5PzjQ6AhqtJ6yjXVQezJ+q6qdF5G8JnqT4xwnL2PEErQzVrb6VWBMVwwopa1nmraXRt9fIKaxpeBw5rRXC1CHKRXrO/dl8U6CD8LtESdqCCrHGtYblsdSyUlWLBZMRKVMKy9c8WVW55HMnGoNbK4SpRVQezCPuz3ubJ05nEbfJdZpZumGWSpA7EjQ3GpzWnMeOj0cmvnlUpvJXs5Rq7YlrTC6iXKRHiLgvVPXKVCTqIOIky3nKJa1B9WEB0jB3BMr7/oJjKWUzQj6X5VChWNo3bGa0PwM4ylKqpyeuMbmIcpE+4/68GvglnDaZANcDL6coU8cQJ87hqZQ0lEu2Szhy7DgLBzeE9qsNurnXbtw1oQFVcUyZOb27NN5jaHiEr28Jn4/tvXcL3BpRRLlI/w9ARD6lqr/qe+oREbHJjoTHOXrzOY4cO57q8nQ+l+Ww7xwjowVuWbeVm9dtDbUchoZHJlgvfiqHvEdZZ57lZIFbI4o4mbxzROSt3gMRWQjMSU+kzmFgxeLQ7Ni0lIvXsGnm9O4JsRR/pfTN67aydPVjpaZTQ8MjDDy4LbJ1Zpwh7zAxA7i/r5enBy/mpTWX8fTgxaZcjBJxEu1uAb4rIl727gLg91KTqIOI+vZOur2Bx1PP7wfiuWejhSIDD24ryRgVxM1mJNaQ94yIuT9GbOIMXvu2iCwCvFGxz3tjZKMQkfk4EyB/CRgH7lLVz4vIbGAdjqJ6GfiAqh4UEQE+D7wXOAJ8WFV/4B7rBpxMYoDb22llKyzOkY85E6hWvJs+7jJ0cVxZtX5n9XacFbrHYitGElR1kUSkBxgA/khVtwGnicjlMY59HPi4qv534HzgJhE5C6e3zBOqugin7MDrNXMpsMj9dyNu7ZOrkFYC5wHvBFaKyKz4bzGYoeERlq95koWDG1i+5smSK5EUq64821kWThhvBlGQexbGqLsyFEVxXMv6Bof18TXlYtRCHBfpHmALcIH7eC/wIPBo1ItU9TXgNff3X4jIc0AvcBVwobvbvcB3cdpvXgX8o1u5/YyI5EVkrrvv416TKxF5HHgPcH+sdxhAI53/4+IdZ/UjOxONx3irUX73LI4lM7BiMQMPbqupyZMlxRmNEifIe4aqfhooAqhqAUIr7wMRkQU4w9s2AW9xlY+nhE5xd+sF9vhettfdFra98hw3ishmEdm8f//+SHnSGmIfxOshxX/14p9B1N/XG8uSmdXjZNKuff8S8rls6H4KqVhzxtQl1uA1EcnheukicgZQNQbjISIn4Yw8uVlVfy7h80SDntCI7eUbVO8C7gKn2DFKprSG2PtbHqSRWBfUv6XacnI2I6y8wsmk9SySqAzkNKw5Y+oSR8GsBL4NzBeR+3BmUn84zsFFJIujXO5T1YfdzT8Rkbmq+prrAv3U3b4XmO97+anAPnf7hRXbvxvn/GEkMcQ+qC2Dv+gvCeUyqydLz7TuyPySKKUYlg9Tzb2ypttGUkQqGHdl53mcbN7zcayJj6nqz6od2H3t3cBzqvpZ31PrgaioZPsAAAr2SURBVBuANe7Pb/i2/5GIfA0noHvIVUIbgb/yBXYvAW6N+f4CaXSIfVAMx9+xPwly2Qwrrzi76k0elewXNgcJTlgzCwc3BMptTbeNJIiMwbgB1yFV/U9V3aCqj8ZRLi7Lgf8FXCwiW91/78VRLL8hIi/gzFxa4+7/TZxOebuB/4PTOQ83uPsp4Fn33ycbnWrQ6ApJUIe6ZJVLV2x5opL94hBmtVkTKCMJ4rhIz4jIr6jqs7UcWFX/jfBg8LsD9lfgppBjfQX4Si3nr0a9KyRDwyN1rwp5cZlq8ZljxzW2bI2m6jdqzRlGFHEUzEXA74vIyzgTBQRHH7w9TcHalXpWmioT1Kq1eag1ftPIcrLVEhlpEkfBXJq6FB1ErbGJoNT6atMSM+ErbaFUBp1rURKW72KkRVQ/mBk4Db9/GdgB3K2qyQ5P7kDCgqq5bBcgsVPr+/t6Q/utXH/e/AnbomhG4qBh1ENUkPdeYBmOcrkU+JumSNTmhAVV77j67RMCx+97Ry9rN+4KLUe4vf8cPnT+aSWLJePOD7q9/5yaZGpm4mDapF3CYTSXKBfpLFU9B0BE7ga+3xyR2ptqMYuwOEuYVXF7/zk1K5RK0kocbDZmiU0+ohRMaalEVY9HZOBOOeqdPZ1WAlsSiYPtQDOvmdEcolykJSLyc/ffL4C3e7+LyM+bJWCncdvQDs649ZuhBYhpWBWN5sK0C5PFEjNOENUyM14vAKPEbUM7AoO2ftKwKibLUvNkscSME9gQ+wS5f9OeyOfTtComw1KzJf1NPkzBJEhUgpyN8KjOZLHEjBOYgmkQf4JbGBkRnh68uLQEW3nzNJIkN9mYDJaYcQJTMA0Qd7Lj9efND12C3fzKgbI2D7Y0a0wm4nS0M0Ko1uzJnzgXtgR7/6Y9kyZJzjAqMQumRuIMuxfgpTWXlW0Lc6HC4jZB+5srZXQapmBqIK5LFLSsGjVnKEjJVB7DslyNTsRcpBqIM+w+bFk1LBnu+vPmx0qSm0z1RsbUwSyYGohaKRIouS1A4GoRBC/BLjt9dlXXx7JcjU7EFEwNxOl/W82VCWvdUG/vXctyNdoZc5FqIE7NT1quzGSpNzKmFmbB1ECcTNO0XBnLcjU6EVMwNVLNnUnTlbEsV6PTMBcpYcyVMYwTmAWTMObKGMYJTMGkgLkyhuFgLpJhGKlhCsYwjNQwBWMYRmqYgjEMIzVMwRiGkRodo2BE5D0isktEdovIYKvlMQyjOh2hYEQkA/w9zgjbs4DrReSs1kplGEY1OkLBAO8Edqvqi6p6DPgacFUjBxwfH0cjpgAYhtE4naJgegH/0KG97ra6GRsbY9OmTQ0JZRhGNJ2iYIIGY5eZHyJyo4hsFpHN+/fvr3rAbDbL0qVLk5LPMIwAOkXB7AXm+x6fCuzz76Cqd6nqMlVdNmfOnFgHnTFjRnISGoYxgU5RMM8Ci0RkoYhMA64D1rdYJsMwqtARxY6qelxE/gjYCGSAr6jqzmac20aFGEb9dISCAVDVbwLfbOY5bVSIYTRGxyiYVhDVX7cdFIxZV0a7YwomgrA+ukEtMZuNWVdGJ9ApQd6WENZHV3Bu8FZig9iMTsAUTAQDKxaHJuC0+ka2QWxGJ2AKJoL+vt7QAfetvpHDrCsbxGa0E6ZgqtDbpjeyTS8wOgFTMFVo1xu5v6+XO64+h958DsFRhHdcfY4FeI22wlaRqtDOY0hseoHR7piCiYHdyIZRH+YiGYaRGqZgDMNIDVMwhmGkhkzGtpEish94JcaubwZ+lrI4aWByNxeTuzqnq+qERkyTUsHERUQ2q+qyVstRKyZ3czG568dcJMMwUsMUjGEYqTHVFcxdrRagTkzu5mJy18mUjsEYhpEuU92CMQwjRUzBGIaRGlNWwYjIe0Rkl4jsFpHBNpBnvog8JSLPichOEfmYu322iDwuIi+4P2e520VEvuDKv11EzvUd6wZ3/xdE5IYmyJ4RkWERedR9vFBENrnnX+eOmkFEpruPd7vPL/Ad41Z3+y4RWdEEmfMi8pCIPO9e8ws65Frf4n4+figi94vIjLa+3qo65f7hjD75MfBWYBqwDTirxTLNBc51f38D8O/AWcCngUF3+yDw1+7v7wW+hdPB83xgk7t9NvCi+3OW+/uslGX/E+CfgUfdxw8A17m/fwn4A/f3PwS+5P5+HbDO/f0s928wHVjo/m0yKct8L/C77u/TgHy7X2uccckvATnfdf5wO1/vlt1QrfwHXABs9D2+Fbi11XJVyPgN4DeAXcBcd9tcYJf7+5eB633773Kfvx74sm972X4pyHkq8ARwMfCoexP+DOiuvNY4c60ucH/vdveTyuvv3y8lmd/o3qhSsb3dr7U3o322e/0eBVa08/Weqi6S94fy2OtuawtcU7YP2AS8RVVfA3B/nuLuFvYemv3e7gT+FBh3H78JGFXV4wHnL8nmPn/I3b/ZMr8V2A/c47p2/yAiM2nza62qI8BngFeB13Cu3xba+HpPVQUT1su75YjIScDXgZtV9edRuwZs04jtiSMilwM/VdUtMeSKeq7Zf49u4Fzgi6raBxzGcYnCaAu53ZjQVThuzTxgJnBphAwtl3uqKpi9wHzf41OBfS2SpYSIZHGUy32q+rC7+SciMtd9fi7wU3d72Hto5ntbDlwpIi8DX8Nxk+4E8iLiNTPzn78km/v8ycCBJsvsybFXVTe5jx/CUTjtfK0Bfh14SVX3q2oReBh4F218vaeqgnkWWORG36fhBMDWt1IgERHgbuA5Vf2s76n1gLc6cQNObMbb/lvuCsf5wCHXrN8IXCIis9xvvEvcbYmjqreq6qmqugDnGj6pqh8EngKuCZHZey/XuPuru/06d9VjIbAI+H4aMrty/wewR0S8xsrvBn5EG19rl1eB80Wkx/28eHK37/VOKyDV7v9wVgb+HSeC/udtIM//wDFTtwNb3X/vxfGZnwBecH/OdvcX4O9d+XcAy3zH+m1gt/vvI02S/0JOrCK91f3A7gYeBKa722e4j3e7z7/V9/o/d9/LLuDSJsi7FNjsXu8hnFWgtr/WwGrgeeCHwD/hrAS17fW2UgHDMFJjqrpIhmE0AVMwhmGkhikYwzBSwxSMYRipYQrGMIzUsMmORixExFvCBfglYAwn3R7gnap6rAUybQSuUdVfNPvcRjxsmdqoGRFZBfyXqn6mYrvgfKbGA1+Y3Pmbch6jccxFMhpCRH7Z7U3yJeAHwHwRGfU9f52I/IP7+1tE5GER2Swi33ezYiuP97si8i8istHtVXJbyHnmisheEcm7z3/E7dWyTUTuiXs+I13MRTKS4CycLNbf99XEBPEF4NOq+oxbMf4o8LaA/d7pbj8GPCtOI6v/8p8HwDFkQESWAJ8A3qWqB0Rkdo3nM1LCFIyRBD9W1Wdj7PfrwGJPMQCzRCSnqoWK/Taq6kEAERnCKaP4dsR5LsZppnQAwPtZw/mMlDAFYyTBYd/v45S3A5jh+12IFxCuDAx6jw9X7ug7blAwMe75jJSwGIyRKG7g9aCILBKRLuA3fU9/B7jJeyAiS0MOc4k4PXN7cPqfPF3ltN/BqQ6e7R53tm97nPMZKWEKxkiDT+C4NE/g9B7xuAlY7gZjfwR8NOT1/4bT43cYuF9Vt0adTFW34/TT/VcR2QqsrfF8RkrYMrXRVojI7wJvU9WbWy2L0ThmwRiGkRpmwRiGkRpmwRiGkRqmYAzDSA1TMIZhpIYpGMMwUsMUjGEYqfH/Adu2O/SFLGMbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 288x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#还可以观察预测值与真值的散点图\n",
    "plt.figure(figsize=(4, 3))\n",
    "plt.scatter(y_train, y_train_pred_lr)\n",
    "plt.plot([-3, 3], [-3, 3], '--k')   #数据已经标准化，3倍标准差即可\n",
    "plt.axis('tight')\n",
    "plt.xlabel('True price')\n",
    "plt.ylabel('Predicted price')\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在y的真值大的部分预测效果不好"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 正则化的线性回归（L2正则 --> 岭回归）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "语法：  \n",
    " _BaseRidgeCV(alphas=(0.1, 1.0, 10.0), fit_intercept=True, normalize=False, scoring=None, cv=None, gcv_mode=None, store_cv_values=False)  sklearn.linear_model.ridge模块中出现的类。  \n",
    "参数名：alphas  \n",
    "类型： numpy array of shape [n_alphas]  \n",
    "说明：α值的数组。正则化的力度，必须是正浮点数。正则化提升了问题的条件，减少了估计器的方差。较大的值指定了更强的正则化。在其他模型，比如   LogisticRegression 或者LinearSVC，α对应C−1（-1为上标）。  \n",
    "\n",
    "#### 参数\n",
    "参数名：fit_intercept  \n",
    "类型：boolean  \n",
    "说明：是否计算该模型的截距。如果设置为False，将不会在计算中使用截距（比如，预处理数据已经中心化）  \n",
    "\n",
    "参数名：normalize  \n",
    "类型：boolean, optional, default False   \n",
    "说明：当fit_intercept设置为False时，该参数将会被忽略。如果为True，则回归前，回归变量X将会进行归一化，减去均值，然后除以L2范数。如果想要标准化，请在评估器（normalize参数为False）调用fit方法前调用sklean.preprocessing.StandardScaler，  \n",
    "\n",
    "参数名：scoring  \n",
    "类型：string, callable or None, optional, default: None  \n",
    "说明：一个字符串(见模型评估文档)或一个评估分数的可调用的对象/函数，对象/函数带有注册的评估器(estimator, X, y)   \n",
    "\n",
    "Regression   \n",
    "‘explained_variance’ metrics.explained_variance_score  \n",
    "‘neg_mean_absolute_error’ metrics.mean_absolute_error  \n",
    "‘neg_mean_squared_error’ metrics.mean_squared_error  \n",
    "‘neg_mean_squared_log_error’ metrics.mean_squared_log_error  \n",
    "‘neg_median_absolute_error’ metrics.median_absolute_error ‘r2’ metrics.r2_score  \n",
    "\n",
    "参数名：cv  \n",
    "类型：int, cross-validation generator or an iterable, optional  \n",
    "说明：确定交叉验证分裂策略。对cv的可能输入如下:   \n",
    "None：使用有效的留一交叉验证。  \n",
    "integer：指定折叠数。  \n",
    "-：用于交叉验证生成器的对象。  \n",
    "-：一种可重复的生成序列。  \n",
    "对于整数/无输入，如果y是二进制或多类，使用sklearn.model_selection.StratifiedKFold，否则使用sklearn.model_selection.KFold  \n",
    "参考用户指南来的得到这里使用的各种交叉验证策略  \n",
    "\n",
    "参数名：gcv_mode  \n",
    "类型： {None, ‘auto’, ‘svd’, eigen’}, optional  \n",
    "说明：说明在执行通用交叉验证时使用的策略的标志。选项有:   \n",
    "auto：如果n_samples为> n_feature，或者当X为稀疏矩阵时，使用svd，否则使用eigen。  \n",
    "svd：用X奇异值分解的力计算(不适用于稀疏矩阵)   \n",
    "eigen：力的计算通过eigen分解 XT（T上标）XXT（T上标）X  \n",
    "“auto”模式是默认的，它的目的是根据训练数据的形状和格式选择两个更廉价的选项。  \n",
    "\n",
    "参数名：store_cv_values  \n",
    "类型：boolean, default=False  \n",
    "说明：标记是否与每个alpha对应的交叉验证值应该存储在cv_values_属性中(见下面)。此标志仅与cv=None兼容(即使用通用交叉验证)。   \n",
    "\n",
    "#### 属性    \n",
    "参数名：cv_values_  \n",
    "类型：array, shape = [n_samples, n_alphas] or shape = [n_samples, n_targets, n_alphas], optional  \n",
    "说明：每个alpha的交叉验证值(如果store_cv_values=True和cv=None)。在fit()被调用之后，这个属性将包含均方差(默认值)或{loss,score}_func函数(如果在  构造函数中提供)。   \n",
    "\n",
    "参数名：coef_  \n",
    "类型： array, shape = [n_features] or [n_targets, n_features]  \n",
    "说明：权重向量   \n",
    "\n",
    "参数名：intercept_  \n",
    "类型：float | array, shape = (n_targets,)  \n",
    "说明：决策函数中的独立项，即截距，如果fit_intercept=False,则设置为0  \n",
    "\n",
    "参数名：alpha_  \n",
    "类型： float  \n",
    "说明：正则化参数估计。  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of RidgeCV on test is 1319.4022139653068\n",
      "The RMSE of RidgeCV on train is 1236.2599118626415\n"
     ]
    }
   ],
   "source": [
    "#岭回归／L2正则\n",
    "#class sklearn.linear_model.RidgeCV(alphas=(0.1, 1.0, 10.0), fit_intercept=True, \n",
    "#                                  normalize=False, scoring=None, cv=None, gcv_mode=None, \n",
    "#                                  store_cv_values=False)\n",
    "from sklearn.linear_model import  RidgeCV\n",
    "\n",
    "#1. 设置超参数（正则参数）范围\n",
    "alphas = [ 0.01, 0.1, 1, 10,100]\n",
    "#n_alphas = 20\n",
    "#alphas = np.logspace(-5,2,n_alphas)\n",
    "\n",
    "#2. 生成一个RidgeCV实例，内置交叉验证的岭回归，默认情况下，它执行通用的交叉验证，这是一种有效的留一交叉验证的形式。\n",
    "ridge = RidgeCV(alphas=alphas, store_cv_values=True, scoring=\"neg_mean_squared_error\")  \n",
    "#3. 模型训练\n",
    "ridge.fit(X1_train, y_train)    \n",
    "\n",
    "#4. 预测\n",
    "y_test_pred_ridge = ridge.predict(X1_test)\n",
    "y_train_pred_ridge = ridge.predict(X1_train)\n",
    "\n",
    "\n",
    "# 评估，使用r2_score评价模型在测试集和训练集上的性能\n",
    "#print ('The r2 score of RidgeCV on test is', r2_score(y_test, y_test_pred_ridge))\n",
    "#print ('The r2 score of RidgeCV on train is', r2_score(y_train, y_train_pred_ridge))\n",
    "\n",
    "#评估,使用RMSE评价模型在测试集和训练集上的性能\n",
    "print ('The RMSE of RidgeCV on test is', np.sqrt(mean_squared_error(y_test, y_test_pred_ridge))) \n",
    "print ('The RMSE of RidgeCV on train is', np.sqrt(mean_squared_error(y_train, y_train_pred_ridge))) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "# print('系数矩阵:\\n',redge.coef_)\n",
    "# print('线性回归模型:\\n',redge)\n",
    "# print('交叉验证最佳alpha值',redge.alpha_)  # 只有在使用RidgeCV算法时才有效"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 可视化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEGCAYAAABsLkJ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXxU5dn/8c9FIOxbFsISMBDWoKwJCrgrlKp1rXUvi4o81tbWX/uota3dl8euVq3SioC4L1SrtgLuFSlJ2An7HiAkEAhrQpb798cMmGICA2TmzMz5vl+vvJjMnMx855DMNec+132POecQERH/aeR1ABER8YYKgIiIT6kAiIj4lAqAiIhPqQCIiPhUY68DnIyUlBSXkZHhdQwRkZiSn5+/0zmXeuz1MVUAMjIyyMvL8zqGiEhMMbNNdV2vISAREZ9SARAR8SkVABERn1IBEBHxKRUAERGfUgEQEfEpFQAREZ+KqXkAIhLb9pVXkruxlLXF+8nJSGJgejsaNTKvY/mWCoCIhE15ZTX5m3Yzd91O5q7bxZLCMqprPv8Mkg6tm3JJvzRG909jRGYyTRsneJjWf1QARKTBVFbXsKRwD3PX7mLuul3kb97N4aoaEhoZA9Lb8j8XZDIiM5meHVoxd90uZhUU8eairbwwfzMtExO4sE8HRmWlcVGfDrRt0cTrpxP3LJY+ESw7O9tpKQiR6FFd41ixfe/Rd/jzN5Ry8HA1AFmd2jAiM5kRPZPJyUiidbO6X9DLK6v5bN0uZhXsYHbBDnbur6BxI+PsHkmMzurIpVlpdGnXPJJPK+6YWb5zLvsL16sAiEionHOsLd7P3HW7mLtuJ/PWl1J2qBKAzNSWjMhMYURmMuf0SKZ9y8STvv+aGseiwj3MLtjBrOVFrCs5AED/zm0YndWRUVlp9OvUGjOdNzgZKgAictKcc2wpPXT0Hf7cdbvYub8CgC7tmjOyZzIjMlMYnplMWptmDf7460r2Mzt4ZLBg826cg/T2zRmVlcborI7kZLSncYKaGU9EBUBEQrJjb3ngBT84jr91zyEAUls3DQzpZAZe9LsmtYhorpJ9Fby3IlAMPlm7k8NVNbRr0YSL+3RgdP80zu+dSotEndasiwqAiNRp94HDzFu/6+iwzpFhl7bNmzC8R2AMf0RmMpmpraJm6OVARRWfrClh1vIdvLeymLJDlSQ2bsR5PVMYlZXGJf3SSG3d1OuYUUMFQESAz3vxj7zDL9i+F4CWiQkM6550dEinX6c2JMRAj35VdQ3zN5YGzxvsYOueQ5jBkG7tg0NFafRIbeV1TE+pAIj4VH29+ImNGzG0W/ujnToD0tvRJMbH051zrNi+L1AMCopYvi1Q3DJTWzK6f+Ak8iAfTj5TARDxicrqGhZv2XN0SGfBpj0crg704g9Mb3u0U2fIGe1p1iS+J15t3XOIOcFi8J/1pVTVOFJbN+XSfoEjg+GZyXG/D0AFQCRuVdc4CrZ93oufuzHQi29Wqxc/M4Wc7km0aurfk6RlByv5YFUxswt28OGqYg4crqZlYgIX9ElldFbHuJ58pgIgEieO14vfs0Oro506Z3c/tV58P6ioqmbuul1HW0xL9gUmnw3rnsTorDRG9e8YV5PPVABEYtTxevHT2zdnZGYKI3omM7xHMh3C0Isf72pqHIsL9xydiby2eD8QmHw2KiuNUVlpZHVqEzUdUKdCBUAkhhSVlfPZ+ujrxfeD9bUmn+UHJ591aRecfNY/jWEZSTE3+UwFQCSKlR7txQ+8y18fA734flCyr4L3VwbaS49MPmvbvAmX9A0sWnd+71RaxsB5FRUAkSiyr7yS+RtKjw7prKinFz+rUxvftSxGq4OHq/h49U5mFRTx/spi9hwMTD479+jksw50aB2dQ3AqACIeOl4vfvYZgV784ZkpDEhvG/O9+H5QVV1D7sbdR+cbFO4OTD4b3LXd0fkGmVE0+UwFQCSCDlcF18Wvoxd/UNd2wRf8ZIZ0i/9e/HjnnGNl0eeTz5ZtDRzN9UhteXQF08FdvZ18pgIgEkbH68Xv37nN0SGdnAx/9+L7wZHJZ7MLdjBv/a5ak88C5w1GZKZEvOirAIiEybvLi7j/tSXsOahefPlvZYcq+XBVMbMKdvDhysDksxaJCVzQO5XR/dO4uE9aRCafRWUBMLMpwBVAsXPuzBNtrwIg0WbRlj3c8NRn9E5rzR3ndVcvvtSrourzTz6bU7CD4n0VJDQyzu6edHS+QXr78LT1RmsBOB/YD0xXAZBYU7j7IFc/PpfmiY34+90jSW6l5YclNDU1jiVby5i1vIjZBTtYE5x8ltWpzdH5Bg05+SwqCwCAmWUAb6kASCzZW17J9X/5jG1lh5h59wh6dmjtdSSJYRt2HmB2QRGzltcx+SwrjZzuSafVHRazBcDMJgITAbp16zZ006ZNkQsnUoeq6homTMtj7tqdTJswjJE9U7yOJHFk5/4K3l9RzKyCIj5Zs5OK4OSzP980mPN7p57SfdZXAKK+HcE5NxmYDIEjAI/jiM855/jxP5bz8eoSfn3tWXrxlwaX0qopX8vpytdyuh6dfDa7YAeZHRp+XkHUFwCRaDLl043MmLeZuy7owY3DunkdR+Jci8TGjDmzI2PO7BiW+9eUQ5EQzS7Ywc/fLmBM/47c/6W+XscROW2eFgAzewH4DOhjZoVmdruXeUTqs2xrGd96YSEDurTlDzcM0vo8Ehc8HQJyzt3k5eOLhGJ72SFun5ZLUstE/jo2m+aJWrpB4oPOAYgcx4GKKm6fmseBimpe/Z9hUbvao8ip0DkAkXpU1zjufXEhK4v28tjNg+nbsY3XkUQalI4AROrxi7dXMGdFMT+7qj8X9ungdRyRBqcjAJE6PPvZRqZ8uoEJI7tz2/AMr+OIhIUKgMgxPlxVzMNvLufSfh146PJ+XscRCRsVAJFaVhbt5Z7nF9K3Yxv+dONgEtTuKXFMBUAkqHhfOROeyaVl0wSeHpcdEx/2LXI69BsuAhw6XM2d0/LYfbCSVyYNp1Pb5l5HEgk7FQDxvZoax3deWsSSrWVMvi2bM7u09TqSSERoCEh87zfvruRfy4v4weVZjMpK8zqOSMSoAIivvTh/M099tJ5bz+nGhJEZXscRiSgVAPGtT9fu5Ad/X8YFvVP58Vf6N9jH74nEChUA8aW1xfuYNCOfzNRWPHbzYBqfxsfticQq/daL7+zcX8H4qbk0bRxo92zdrInXkUQ8oQIgvlJeWc3E6XkU763gb2OzSW/fwutIIp5RG6j4Rk2N43uvLmHB5j08ccsQBnVt53UkEU/pCEB8449zVvOPxdu4f0xfLjurk9dxRDynAiC+8Fp+IY++v5Ybsrsy6YIeXscRiQoqABL3/rN+Fw+8voQRmcn87Ooz1e4pEqQCIHFtw84D3DUjn25JLfjLLUNJbKxfeZEj9NcgcWv3gcNMmJpLIzOmjMuhbQu1e4rUpgIgcamiqpq7ZuSzdfchJt82lDOSW3odSSTqqA1U4o5zjgdfX8r8DaX86cZBZGckeR1JJCrpCEDizmPvr+X1BVv5zqW9uWpQF6/jiEQtFQCJK28u3sbvZq/mmsFd+NYlPb2OIxLVVAAkbuRvKuW7ryxmWEYSv77uLLV7ipyACoDEhc27DjJxej6d2zbjqduG0rRxgteRRKKeCoDEvLJDlYyfOp+qGseUcTm0b5nodSSRmKACIDGtsrqGu5/LZ3PpQZ66bSg9Ult5HUkkZqgNVGKWc44f/n0Zn67dxW+vH8g5PZK9jiQSU3QEIDFr8sfreTF3C/dc1JOvDk33Oo5IzFEBkJj0r2Xb+fW/VnLFgE7cN6q313FEYpIKgMScxVv28O2XFjGoazt+e/1AGjVSu6fIqVABkJiydc8h7pieR0qrpvz169k0a6J2T5FTpZPAEjP2lVdy+9Rcyg9X8/wdZ5PSqqnXkURimgqAxISq6hrueX4ha4r3M3V8Dr3SWnsdSSTmeToEZGZjzGyVma01swe8zCLRyznHT/5RwEerS/j51WdyXq9UryOJxAXPCoCZJQCPA18GsoCbzCzLqzwSvZ75dCPPztvExPN7cNOwbl7HEYkbXh4BDAPWOufWO+cOAy8CV3mYR6LQnIId/OztAkZnpfHAmL5exxGJK14WgC7AllrfFwav+y9mNtHM8swsr6SkJGLhxHvLtpbxrRcXcmbntvzxxkFq9xRpYF4WgLr+mt0XrnBusnMu2zmXnZqqsV+/KCor545pebRt3oSnx2bTIlH9CiINzcsCUAh0rfV9OrDNoywSRQ5UVHH7tFz2lVcyZVwOHdo08zqSSFzysgDkAr3MrLuZJQI3Am96mEeiQHWN494XF7Fi+14eu3kI/Tq18TqSSNzy7LjaOVdlZvcA7wIJwBTn3HKv8kh0+OU7K5izYgc/ubI/F/Xt4HUckbjm6cCqc+4d4B0vM0j0eHbeJp7+9wbGjchg7IgMr+OIxD2tBSRR4aPVJfz4zeVc3LcDP7xC00FEIkEFQDy3qmgf33huAb3TWvPoTYNJULunSESoAIiniveVM2FqLi0SE3h6bDatmqrdUyRS9Ncmnjl0uJo7p+dTeuAwL981nM7tmnsdScRXVADEEzU1jvteXsSSwj08eetQzkpv63UkEd/REJB44pFZq/jnsiK+/+V+fKl/R6/jiPiSCoBE3Mu5W/jLh+u4+exu3HFed6/jiPiWCoBE1Ny1O/n+zKWc1yuFn1zZHzN1/Ih4RQVAImZt8X4mzcine0pLHr9lCE0S9Osn4iX9BUpE7NpfwYSpuSQ2bsSUcTm0adbE60givqcuIAm78spqJj6bz4695bww8Ry6JrXwOpKIoAIgYeac439fXUL+pt08fvMQhnRr73UkEQnSEJCE1R/mrOHNxdv43pf6cPmATl7HEZFaVAAkbGYuLOTR99bw1aHp3H1hptdxROQYKgASFvM3lHL/q0s5p0cSv7zmLLV7ikQhFQBpcBt3HmDis3mkt2/Ok7cOJbGxfs1EopH+MqVB7Tl4mAlTczFgyrgc2rVI9DqSiNQj5AJgZuea2fjg5VQz0xx++S+Hq2qYNCOfwt2HmPz1bDJSWnodSUSOI6QCYGYPA/cDDwavagLMCFcoiT3OOR58fSnz1pfyf18dQE5GkteRROQEQj0CuAa4EjgA4JzbBrQOVyiJPU98uI7XFhRy7yW9uHpwF6/jiEgIQi0Ah51zDnAAZqZjeznqrSXbeOTdVVw1qDPfvrSX13FEJEShFoCXzewpoJ2Z3QnMAf4avlgSKxZs3s19Ly8m+4z2/Oa6AWr3FIkhIS0F4Zz7rZmNAvYCfYAfOedmhzWZRL0tpQe5c1oeHds0Y/LXs2nWJMHrSCJyEkIqAMEhn/edc7PNrA/Qx8yaOOcqwxtPolXZoUomTM2lsrqGKeNySGqpdk+RWBPqENDHQFMz60Jg+Gc8MDVcoSS6VVbXcM/zC9iw8wBP3jaUnh1aeR1JRE5BqAXAnHMHgWuBPzvnrgGywhdLopVzjh+9sZxP1uzkl9eexYjMFK8jicgpCrkAmNlw4Bbg7eB1Wkrah/72yQZemL+Zuy/M5GvZXb2OIyKnIdQCcC/wAPC6c255cBbw++GLJdHo3eVF/PKfK7j8rE58d3Qfr+OIyGkK9V38QaAGuMnMbgWM4JwA8YelhWXc++JCBqa343dfG0ijRmr3FIl1oRaA54DvAssIFALxkW17DnH7tFySWzblr2r3FIkboRaAEufcP8KaRKLS/ooqJkzN5dDhambcfTaprZt6HUlEGkioBeBhM/sb8B5QceRK59zrYUklUaGquoZvPr+ANcX7eWZcDr3TtPyTSDwJtQCMB/oSWAX0yBCQA1QA4tjP317BB6tK+MU1Z3J+71Sv44hIAwu1AAx0zp0V1iQSVaZ+uoGpczdyx7ndueXsM7yOIyJhEGob6Dwz08Qvn3h/5Q5++lYBo7LSePCyfl7HEZEwCfUI4FxgrJltIHAOwADnnBsQtmTiieXbyrjn+YVkdW7Dn24cRILaPUXiVqgFYExDPqiZXQ/8GOgHDHPO5TXk/cupKSor5/apebRt3oSnx+bQIlGTvUXiWajLQW9q4MddRmBdoaca+H7lFB2oqOL2abnsK6/klUkjSGvTzOtIIhJmnrzFc86tAPThIVGiusZx74sLWbF9L0+PzSGrcxuvI4lIBIR6EtgzZjbRzPLMLK+kpMTrOHHpF2+vYM6KYh7+Sn8u6tvB6zgiEiFhOwIwszlAxzpuesg590ao9+OcmwxMBsjOztb6Qw1s+mcbmfLpBsaPzGDsiAyv44hIBIWtADjnLg3XfUvD+GBlMT9+czmX9uvADy5Xl6+I30T9EJCER8G2vdzz/AL6dWrDn24crHZPER/ypACY2TVmVggMB942s3e9yOFXO/aWc/u0XFo3C7R7tmyqdk8RP/KqC2gmMNOLx/a7g4cD7Z5lhyp5ZdJwOrZVu6eIX2kIyEcC7Z6LKNi2lz/fNJj+ndt6HUlEPKQC4CO/emcFswt28MMrsrikX5rXcUTEYyoAPvHsvE387d8bGDcig/Eju3sdR0SigAqAD3y4KtDueXHfDvzwCrV7ikiACkCcW1m0l3ueX0jvtNY8epPaPUXkcyoAcax4bzkTnsmlZdMEpozLppXaPUWkFr0ixKmDh6u4Y3oeuw8G2j07tW3udSQRiTIqAHGopsbxnZcWsXRrGZNvy+bMLmr3FJEv0hBQHPr1v1by7vId/PDyLEZlqd1TROqmAhBnnvvPJiZ/vJ6vDz+D8SMzvI4jIlFMBSCOfLS6hB+9sZwL+6Tyoyuy9IE7InJcKgBxYlXRPr7x3AJ6dWjFYzcPoXGC/mtF5Pj0KhEHiveVM2FqLi0SE5gyLkftniISEr1SxLhDh6u5c1oepQcO8/Jdw+ncTu2eIhIaFYAYVlPjuO/lRSzZWsZTtw7lrHS1e4pI6DQEFMN+8+5K/rmsiIcu68fo/nV9/LKISP1UAGLUC/M389RH67n1nG7cfq5W9xSRk6cCEIM+WVPCD/6+jAt6p/Ljr/RXu6eInBIVgBizesc+7p5xpN1zsNo9ReSU6dUjhpTsq2D8M7k0S0zg6XE5tG7WxOtIIhLDVABiRHllNXdOz2PXgQqeHptNF7V7ishpUhtoDDjS7rm4cA9/uWUoA9LbeR1JROKAjgBiwCOzVvHO0iK+/+V+jDlT7Z4i0jBUAKLcS7mb+cuH67j57G7ccZ7aPUWk4agARLFP1+7koZnLOK9XCj+5Uu2eItKwVACi1Jod+5g0I58eqS15/JYhNFG7p4g0ML2qRKGd+ysYPzWXpo0Dq3u2UbuniISBCkCUOdLuuXN/BX8bm016+xZeRxKROKU20ChSU+P4f68sZtGWPTxx8xAGdVW7p4iEj44AosjvZq/i7SXbeWBMX758Viev44hInFMBiBIv523h8Q/WcdOwrkw8v4fXcUTEB1QAosDctTv5/utLOa9XCj+96ky1e4pIRKgAeGxt8X4mzcine4raPUUksvRq46Fd+ysYP3U+iY0bqd1TRCJOXUAeKa+sZuKz+RTvreDFiefQNUntniISWSoAHqipcXzv1SXkb9rNE7cMYXC39l5HEhEf0hCQB/4wZzX/WLyN+8f05TK1e4qIRzwpAGb2iJmtNLMlZjbTzHwz4+nV/EL+/P5absjuyqQL1O4pIt7x6ghgNnCmc24AsBp40KMcEfXZul08+PoSRvZM5ufXqN1TRLzlSQFwzs1yzlUFv50HpHuRI5LWlQTaPc9IbskTtwxVu6eIeC4aXoUmAP+s70Yzm2hmeWaWV1JSEsFYDaf0wGEmTM2lcSPjmXE5tG2udk8R8V7YuoDMbA5Q1+cXPuSceyO4zUNAFfBcfffjnJsMTAbIzs52YYgaVuWV1UycnkdRWTkvqN1TRKJI2AqAc+7S491uZmOBK4BLnHMx98IeCucc97+2hLxNu3n85iEMUbuniEQRT+YBmNkY4H7gAufcQS8yRMIf5qzhjUXb+N6X+nD5ALV7ikh08eocwGNAa2C2mS0ysyc9yhE2r+UX8uh7a/hadjp3X5jpdRwRkS/w5AjAOdfTi8eNlHnrd/HA60sY3iOZn199lto9RSQqRUMXUFxZX7Kfu57Np1tSC568dSiJjbWLRSQ66dWpAf13u+cw2rZQu6eIRC8tBtdAKqqquevZPLaVlfPCnefQLVntniIS3XQE0ACcczzw2lJyN+7md9cPZOgZavcUkeinAtAA/vTeGmYu3Mp3R/fmKwM7ex1HRCQkKgCnaebCQv44Zw3XDUnnGxfFdXOTiMQZFYDTMH9DKfe/upRzeiTxq2vV7ikisUUF4BRt2HmAic/mkd6+udo9RSQm6VXrFOwOtns2MuOZ8Tm0a5HodSQRkZOmAnCSKqqquWtGPlt3H2LybUM5I7ml15FERE6J5gGcBOccD762lPkbSvnTjYPIzkjyOpKIyCnTEcBJ+PP7a3l94VbuG9WbqwZ18TqOiMhpUQEI0RuLtvL72au5dkgXvnmx2j1FJPapAIQgd2Mp33tlCcO6q91TROKHCsAJbNx5gInTA+2ek28bStPGCV5HEhFpECoAx7HnYKDdE2DKOLV7ikh8URdQPQ5X1TBpRj6Fuw/x3J1nk5Gidk8RiS8qAHVwzvHg60uZt76UP94wiBy1e4pIHNIQUB0e/2Atry0o5NuX9uLqwWr3FJH4pAJwjDcXb+O3s1ZzzeAu3HtJL6/jiIiEjQpALfmbSvnuK4sZlpHEr69Tu6eIxDcVgKBNuw5w5/R8urRrzlNq9xQRH1ABAMoOVjJ+ai41zjFlXA7tW6rdU0Tin+8LwJF2zy2lB3nq1qF0V7uniPiEr9tAnXM8NHMpn63fxR9uGMjZPZK9jiQiEjG+PgJ44sN1vJJfyLcu6cU1g9O9jiMiElG+LQBvLdnGI++u4qpBnfnOpWr3FBH/8WUByN+0m/teXkxORnt+c90AtXuKiC/5rgBs3nWQidPz6NS2GU/dlk2zJmr3FBF/8lUBKDtUyfip86mqcTwzLocktXuKiI/5pgBUVtdw93P5bC49yFO3DaVHaiuvI4mIeMoXbaDOOX4wcxmfrt3F764fyDlq9xQR8ccRwJMfreelvC188+KeXDdU7Z4iIuCTApDevjnXD03nvlG9vY4iIhI1fDEE9JWBnfnKwM5exxARiSq+OAIQEZEv8qQAmNnPzGyJmS0ys1lmprfnIiIR5tURwCPOuQHOuUHAW8CPPMohIuJbnhQA59zeWt+2BJwXOURE/Myzk8Bm9gvg60AZcNFxtpsITATo1q1bZMKJiPiAOReeN99mNgfoWMdNDznn3qi13YNAM+fcwye6z+zsbJeXl9eAKUVE4p+Z5Tvnso+9PmxHAM65S0Pc9HngbeCEBUBERBqOV11AtRfgvxJY6UUOERE/C9sQ0HEf1Ow1oA9QA2wCJjnntobwcyXB7U9FCrDzFH82nJTr5CjXyVGukxOtueD0sp3hnEs99kpPCoAXzCyvrjEwrynXyVGuk6NcJydac0F4smkmsIiIT6kAiIj4lJ8KwGSvA9RDuU6Ocp0c5To50ZoLwpDNN+cARETkv/npCEBERGpRARAR8am4LQBm9oiZrQwuOz3TzNrVs90YM1tlZmvN7IEI5LrezJabWY2Z1dvSZWYbzWxpcMnssK9/cRK5Ir2/ksxstpmtCf7bvp7tqoP7apGZvRnGPMd9/mbW1MxeCt7+HzPLCFeWk8w1zsxKau2jOyKUa4qZFZvZsnpuNzN7NJh7iZkNiZJcF5pZWa39FfYVi82sq5l9YGYrgn+L99axTcPuL+dcXH4Bo4HGwcu/AX5TxzYJwDqgB5AILAaywpyrH4FJcB8C2cfZbiOQEsH9dcJcHu2v/wMeCF5+oK7/x+Bt+yOwj074/IG7gSeDl28EXoqSXOOAxyL1+1Trcc8HhgDL6rn9MuCfgAHnAP+JklwXAm9FeF91AoYEL7cGVtfx/9ig+ytujwCcc7Occ1XBb+cBdX0a/DBgrXNuvXPuMPAicFWYc61wzq0K52OcihBzRXx/Be9/WvDyNODqMD/e8YTy/GvnfRW4xMwsCnJ5wjn3MVB6nE2uAqa7gHlAOzPrFAW5Is45t905tyB4eR+wAuhyzGYNur/itgAcYwKBqnmsLsCWWt8X8sUd7hUHzDKz/OCS2NHAi/2V5pzbDoE/EKBDPds1M7M8M5tnZuEqEqE8/6PbBN+AlAHJYcpzMrkArgsOG7xqZl3DnClU0fw3ONzMFpvZP82sfyQfODh0OBj4zzE3Nej+iukPhQ9lyWkzewioAp6r6y7quO60+2JDXQr7BEY657aZWQdgtpmtDL5r8TJXxPfXSdxNt+D+6gG8b2ZLnXPrTjfbMUJ5/mHZRycQymP+A3jBOVdhZpMIHKVcHOZcofBif4ViAYH1c/ab2WXA34FeJ/iZBmFmrYDXgG+7//7wLGjg/RXTBcCdYMlpMxsLXAFc4oIDaMcoBGq/E0oHtoU7V4j3sS34b7GZzSRwmH9aBaABckV8f5nZDjPr5JzbHjzULa7nPo7sr/Vm9iGBd08NXQBCef5Htik0s8ZAW8I/1HDCXM65XbW+/SuB82LRICy/U6er9guvc+4dM3vCzFKcc2FdKM7MmhB48X/OOfd6HZs06P6K2yEgMxsD3A9c6Zw7WM9muUAvM+tuZokETtqFrYMkVGbW0sxaH7lM4IR2nd0KEebF/noTGBu8PBb4wpGKmbU3s6bByynASKAgDFlCef61834VeL+eNx8RzXXMOPGVBMaXo8GbwNeD3S3nAGVHhvy8ZGYdj5y7MbNhBF4rdx3/p077MQ14GljhnPt9PZs17P6K5FnuSH4BawmMlS0Kfh3pzOgMvFNru8sInG1fR2AoJNy5riFQxSuAHcC7x+Yi0M2xOPi1PFpyebS/koH3gDXBf5OC12cDfwteHgEsDe6vpcDtYczzhecP/JTAGw2AZsArwd+/+UCPcO+jEHP9Kvi7tBj4AOgboVwvANuByuDv1+3AJAJLwENgSOPxYO6lHKczLsK57qm1v+YBIyKQ6VwCwzlLar1uXRbO/aWlIEREfCpuh4BERJrHOMwAAAK/SURBVOT4VABERHxKBUBExKdUAEREfEoFQETEp1QAxBfMbP9p/vyrwVnGx9vmQzvOSqqhbnPM9qlm9q9Qtxc5GSoAIicQXAcmwTm3PtKP7ZwrAbab2chIP7bEPxUA8ZXgDMpHzGyZBT5v4Ybg9Y2C0/2Xm9lbZvaOmX01+GO3UGsGspn9Jbjw3HIz+0k9j7PfzH5nZgvM7D0zS6118/VmNt/MVpvZecHtM8zsk+D2C8xsRK3t/x7MINKgVADEb64FBgEDgUuBR4LLJFwLZABnAXcAw2v9zEggv9b3DznnsoEBwAVmNqCOx2kJLHDODQE+Ah6udVtj59ww4Nu1ri8GRgW3vwF4tNb2ecB5J/9URY4vpheDEzkF5xJYFbMa2GFmHwE5wetfcc7VAEVm9kGtn+kElNT6/mvBJbobB2/LIjB9v7Ya4KXg5RlA7YW9jlzOJ1B0AJoAj5nZIKAa6F1r+2ICS3KINCgVAPGb+j6c5Xgf2nKIwBo/mFl34LtAjnNut5lNPXLbCdRec6Ui+G81n/8NfofAGkwDCRyZl9favlkwg0iD0hCQ+M3HwA1mlhAclz+fwKJt/ybwgSmNzCyNwEcCHrEC6Bm83AY4AJQFt/tyPY/TiMBqoAA3B+//eNoC24NHILcR+JjHI3oTHavBSpzREYD4zUwC4/uLCbwr/1/nXJGZvQZcQuCFdjWBT2IqC/7M2wQKwhzn3GIzW0hgpcj1wKf1PM4BoL+Z5Qfv54YT5HoCeM3MriewWueBWrddFMwg0qC0GqhIkJm1coFPgEomcFQwMlgcmhN4UR4ZPHcQyn3td861aqBcHwNXOed2N8T9iRyhIwCRz71lZu2AROBnzrkiAOfcITN7mMBnr26OZKDgMNXv9eIv4aAjABERn9JJYBERn1IBEBHxKRUAERGfUgEQEfEpFQAREZ/6/6a3hBd2i1BxAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "alpha is: 1.0\n"
     ]
    }
   ],
   "source": [
    "mse_mean = np.mean(ridge.cv_values_, axis = 0)\n",
    "plt.plot(np.log10(alphas), mse_mean.reshape(len(alphas),1)) \n",
    "\n",
    "#这是为了标出最佳参数的位置，不是必须\n",
    "#plt.plot(np.log10(ridge.alpha_)*np.ones(3), [0.28, 0.29, 0.30])\n",
    "\n",
    "plt.xlabel('log(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show()\n",
    "\n",
    "print ('alpha is:', ridge.alpha_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef_lr</th>\n",
       "      <th>coef_ridge</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>workingday</td>\n",
       "      <td>3.800799e+16</td>\n",
       "      <td>240.630984</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>holiday</td>\n",
       "      <td>3.800799e+16</td>\n",
       "      <td>-345.294456</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_4</td>\n",
       "      <td>1.917491e+16</td>\n",
       "      <td>740.784787</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_2</td>\n",
       "      <td>1.917491e+16</td>\n",
       "      <td>233.156303</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_3</td>\n",
       "      <td>1.917491e+16</td>\n",
       "      <td>-315.111870</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>season_1</td>\n",
       "      <td>1.917491e+16</td>\n",
       "      <td>-658.829219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>temp</td>\n",
       "      <td>4.984000e+03</td>\n",
       "      <td>2792.513136</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>atemp</td>\n",
       "      <td>5.540000e+02</td>\n",
       "      <td>2026.155271</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-1.706750e+03</td>\n",
       "      <td>-1412.035016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>hum</td>\n",
       "      <td>-2.898500e+03</td>\n",
       "      <td>-2309.435888</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>-1.830830e+16</td>\n",
       "      <td>844.541521</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>-1.830830e+16</td>\n",
       "      <td>460.963238</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-1.830830e+16</td>\n",
       "      <td>-1305.504759</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>-5.589601e+16</td>\n",
       "      <td>162.522076</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>-5.589601e+16</td>\n",
       "      <td>-57.858604</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>773.382980</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>422.254557</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>220.040824</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>112.652848</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-161.627991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>59.058734</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-130.514099</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-198.831620</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-198.153746</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-39.138426</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-279.213985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-579.910076</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "      <td>78.249508</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "      <td>32.423857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "      <td>18.399871</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "      <td>-18.339437</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "      <td>-215.397270</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns       coef_lr   coef_ridge\n",
       "31    workingday  3.800799e+16   240.630984\n",
       "30       holiday  3.800799e+16  -345.294456\n",
       "3       season_4  1.917491e+16   740.784787\n",
       "1       season_2  1.917491e+16   233.156303\n",
       "2       season_3  1.917491e+16  -315.111870\n",
       "0       season_1  1.917491e+16  -658.829219\n",
       "26          temp  4.984000e+03  2792.513136\n",
       "27         atemp  5.540000e+02  2026.155271\n",
       "29     windspeed -1.706750e+03 -1412.035016\n",
       "28           hum -2.898500e+03 -2309.435888\n",
       "16  weathersit_1 -1.830830e+16   844.541521\n",
       "17  weathersit_2 -1.830830e+16   460.963238\n",
       "18  weathersit_3 -1.830830e+16 -1305.504759\n",
       "25     weekday_6 -5.589601e+16   162.522076\n",
       "19     weekday_0 -5.589601e+16   -57.858604\n",
       "12        mnth_9 -6.237891e+16   773.382980\n",
       "13       mnth_10 -6.237891e+16   422.254557\n",
       "6         mnth_3 -6.237891e+16   220.040824\n",
       "11        mnth_8 -6.237891e+16   112.652848\n",
       "4         mnth_1 -6.237891e+16  -161.627991\n",
       "8         mnth_5 -6.237891e+16    59.058734\n",
       "7         mnth_4 -6.237891e+16  -130.514099\n",
       "15       mnth_12 -6.237891e+16  -198.831620\n",
       "5         mnth_2 -6.237891e+16  -198.153746\n",
       "9         mnth_6 -6.237891e+16   -39.138426\n",
       "14       mnth_11 -6.237891e+16  -279.213985\n",
       "10        mnth_7 -6.237891e+16  -579.910076\n",
       "22     weekday_3 -9.390400e+16    78.249508\n",
       "24     weekday_5 -9.390400e+16    32.423857\n",
       "21     weekday_2 -9.390400e+16    18.399871\n",
       "23     weekday_4 -9.390400e+16   -18.339437\n",
       "20     weekday_1 -9.390400e+16  -215.397270"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names1), \"coef_lr\":list((lr.coef_.T)), \"coef_ridge\":list((ridge.coef_.T))})\n",
    "fs.sort_values(by=['coef_lr'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 正则化的线性回归（L1正则 --> Lasso）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of RidgeCV on test is 1308.6501399118395\n",
      "The RMSE of RidgeCV on train is 1242.2591826132964\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\hp\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_split.py:1978: FutureWarning: The default value of cv will change from 3 to 5 in version 0.22. Specify it explicitly to silence this warning.\n",
      "  warnings.warn(CV_WARNING, FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "#### Lasso／L1正则\n",
    "# class sklearn.linear_model.LassoCV(eps=0.001, n_alphas=100, alphas=None, fit_intercept=True, \n",
    "#                                    normalize=False, precompute=’auto’, max_iter=1000, \n",
    "#                                    tol=0.0001, copy_X=True, cv=None, verbose=False, n_jobs=1,\n",
    "#                                    positive=False, random_state=None, selection=’cyclic’)\n",
    "from sklearn.linear_model import LassoCV\n",
    "\n",
    "#1. 设置超参数搜索范围\n",
    "#alphas = [ 0.01, 0.1, 1, 10,100]\n",
    "\n",
    "#2. 生成学习器实例\n",
    "#lasso = LassoCV(alphas=alphas)\n",
    "\n",
    "#1. 设置超参数搜索范围\n",
    "#Lasso可以自动确定最大的alpha，所以另一种设置alpha的方式是设置最小的alpha值（eps） 和 超参数的数目（n_alphas），\n",
    "#然后LassoCV对最小值和最大值之间在log域上均匀取值n_alphas个\n",
    "# np.logspace(np.log10(alpha_max * eps), np.log10(alpha_max),num=n_alphas)[::-1]\n",
    "\n",
    "#2 生成LassoCV实例（默认超参数搜索范围）\n",
    "lasso = LassoCV()  \n",
    "\n",
    "#3. 训练（内含CV）\n",
    "lasso.fit(X1_train, y_train)  \n",
    "\n",
    "#4. 测试\n",
    "y_test_pred_lasso = lasso.predict(X1_test)\n",
    "y_train_pred_lasso = lasso.predict(X1_train)\n",
    "\n",
    "\n",
    "# 评估，使用r2_score评价模型在测试集和训练集上的性能\n",
    "#print 'The r2 score of LassoCV on test is', r2_score(y_test, y_test_pred_lasso)\n",
    "#print 'The r2 score of LassoCV on train is', r2_score(y_train, y_train_pred_lasso)\n",
    "#评估,使用RMSE评价模型在测试集和训练集上的性能\n",
    "print ('The RMSE of RidgeCV on test is', np.sqrt(mean_squared_error(y_test, y_test_pred_lasso))) \n",
    "print ('The RMSE of RidgeCV on train is', np.sqrt(mean_squared_error(y_train, y_train_pred_lasso))) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZ4AAAEICAYAAABvQ5JRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXxV9Z3/8dcnO/saNWwGJCjggpoiSN2tou2IndFK21F0bBkdnZ+dn/2NdpkytZ352TrVGW21Y6sjtq4/tYqKRSq4VsCAoKwSkDUIgbCT7d58fn+cb/CSJiQs994s7+fD+8i5n/P9nvM53JhPzvd8c465OyIiIqmSke4ERESkY1HhERGRlFLhERGRlFLhERGRlFLhERGRlFLhERGRlEpa4TGzPDObZ2aLzGyJmf04xN8xs4XhVWZmL4b4+Wa2M2HdjxK2Nd7MVphZqZndmRAfbGZzzWylmT1jZjkhnhvel4b1hQl9vhfiK8zs0mQdv4iINC4riduuBi509z1mlg28a2avufs59Q3M7HngpYQ+77j7VxI3YmaZwK+ALwEbgA/MbJq7LwV+Btzn7k+b2a+BG4GHwtft7j7UzCaGdteY2QhgIjAS6Af8ycyGuXu8qYPo27evFxYWHuE/hYhIxzJ//vyt7p7f2LqkFR6P/jJ1T3ibHV77/1rVzLoBFwI3NLOp0UCpu68O/Z4GJpjZstD/G6HdVOBfiQrPhLAM8BzwSzOzEH/a3auBT82sNGz//aZ2XlhYSElJSQuOWERE6pnZ2qbWJfUaj5llmtlCYAsw093nJqz+KvCGu+9KiI0NQ3OvmdnIEOsPrE9osyHE+gA73D3WIH5An7B+Z2jf1LZERCRFklp43D3u7qOAAcBoMzs5YfXXgacS3i8Ajnf304AHgBdD3Brb9EHih9tnPzObbGYlZlZSXl7eSBcRETlcKZnV5u47gDeB8QBm1odoiOvVhDa73H1PWJ4OZJtZX6KzkoEJmxsAlAFbgZ5mltUgTmKfsL4HUHGQbTXM92F3L3b34vz8RocoRUTkMCVzVlu+mfUMy52Ai4HlYfXVwCvuXpXQ/rhwHQYzGx1y2wZ8ABSFGWw5RJMDpoVrSLOBq8ImJvH5RIVp4T1h/azQfhowMcx6GwwUAfOO/tGLiEhTkjmrrQCYGmalZQDPuvsrYd1E4O4G7a8CbjazGFAJTAzFImZmtwIzgEzgUXdfEvrcATxtZj8FPgQeCfFHgN+FyQMVYX+4+xIzexZYCsSAWw42o01ERI4+02MRDq64uNg1q01E5NCY2Xx3L25sne5cICIiKaXCIyIif+HRdz9lxpLPkrJtFR4RETlAZU2cX7y+glnLtiRl+yo8IiJygJnLNrO3Js6E0/slZfsqPCIicoCXPtxIQY88xgzuk5Ttq/CIiMh+FXtreOuTcq4Y1Y+MjMZu9nLkVHhERGS/Vz8qI1bnXDkqebexVOEREZH9/vDhRk46rhvDC7onbR8qPCIiAsDabXtZsG4HV56e3Jv2q/CIiAgALy0swwyuOC05s9nqqfCIiAgALy3cyFmDe9OvZ6ek7keFR0REWF+xj1Xle7l05HFJ35cKj4iI8M7KrQCcU5T8Z5Cp8IiICO+WllPQI48T8rskfV8qPCIiHVy8znmvdBtfHNqX8DzOpFLhERHp4BZv3MnOylq+WNQ3JftL5qOv88xsnpktMrMlZvbjEH/MzD41s4XhNSrEzczuN7NSM/vIzM5I2NYkM1sZXpMS4mea2cehz/0Jj87ubWYzQ/uZZtaruX2IiHRU76wsB2Dc0DZeeIBq4EJ3Pw0YBYw3szFh3f9x91HhtTDELgOKwmsy8BBERQSYApwFjAam1BeS0GZyQr/xIX4n8Ia7FwFvhPdN7kNEpCN7Z+VWRhR0p2/X3JTsL2mFxyN7wtvs8DrYc7YnAI+HfnOAnmZWAFwKzHT3CnffDswkKmIFQHd3f9+j53c/DlyZsK2pYXlqg3hj+xAR6ZD2VsdYsG475wxLzdkOJPkaj5llmtlCYAtR8ZgbVv1bGOq6z8zqS2x/YH1C9w0hdrD4hkbiAMe6+yaA8PWYZvYhItIhzfu0gtq4c87Q5E+jrpfUwuPucXcfBQwARpvZycD3gJOALwC9gTtC88amUvhhxA+mRX3MbLKZlZhZSXl5eTObFBFpu95eWU5uVgbFhb2ab3yUpGRWm7vvAN4Exrv7pjDUVQ38D9F1G4jOPgYmdBsAlDUTH9BIHGBz/RBa+Fr//NamttUw34fdvdjdi/PzU/dbgIhIqr27ciujB/cmLzszZftM5qy2fDPrGZY7ARcDyxMKghFde1kcukwDrgszz8YAO8Mw2QzgEjPrFSYVXALMCOt2m9mYsK3rgJcStlU/+21Sg3hj+xAR6XA276pi5ZY9KZvNVi8ridsuAKaaWSZRgXvW3V8xs1lmlk807LUQuCm0nw5cDpQC+4AbANy9wsx+AnwQ2t3l7hVh+WbgMaAT8Fp4AdwNPGtmNwLrgKsPtg8RkY7ovdLoNjlfbC+Fx90/Ak5vJH5hE+0duKWJdY8CjzYSLwFObiS+DbjoUPYhItLRvFu6lV6dsxmRxIe+NUZ3LhAR6YDcnfdKt3L20L5kZCT/NjmJVHhERDqgVeV72LyrOuXDbKDCIyLSIb27Mj3Xd0CFR0SkQ3q3dBuDendmYO/OKd+3Co+ISAcTi9cxZ/W2lE+jrqfCIyLSwSzasJM91bG0DLOBCo+ISIfzXulWzGDsCX3Ssn8VHhGRDubd0q2M7Ned3l1y0rJ/FR4RkQ5kb3WMD9dtZ9wJ6RlmAxUeEZEO5Z2V5dTGnfNPPKb5xkmiwiMi0oHMXLqFHp2yU/oYhIZUeEREOoh4nTNr+WYuODGf7Mz0/fhX4RER6SAWrNvO9n21XDzi2LTmocIjItJB/GnpZrIzjfOGpfcBlyo8IiIdxMxlmxkzpA/d8rLTmocKj4hIB7CqfA+ry/fypTQPs0FyH32dZ2bzzGyRmS0xsx+H+BNmtsLMFpvZo2aWHeLnm9lOM1sYXj9K2Nb40KfUzO5MiA82s7lmttLMnjGznBDPDe9Lw/rChD7fC/EVZnZpso5fRKQ1eWPZZgAuGt6OCw9QDVzo7qcBo4DxZjYGeAI4CTiF6JHV30ro8467jwqvuwDCo7N/BVwGjAC+bmYjQvufAfe5exGwHbgxxG8Etrv7UOC+0I7QbyIwEhgPPBi2LyLSrs1cupkRBd3p37NTulNJXuHxyJ7wNju83N2nh3UOzAMGNLOp0UCpu6929xrgaWCCmRlwIfBcaDcVuDIsTwjvCesvCu0nAE+7e7W7fwqUhu2LiLRb2/ZUM3/t9rTPZquX1Gs8ZpZpZguBLcBMd5+bsC4buBb4Y0KXsWFo7jUzGxli/YH1CW02hFgfYIe7xxrED+gT1u8M7ZvalohIuzV98WfUOYwfeVy6UwGSXHjcPe7uo4jOakab2ckJqx8E3nb3d8L7BcDxYWjuAeDFEG/sYeB+kPjh9tnPzCabWYmZlZSXlzfSRUSk7Xh5YRlFx3RleEG3dKcCpGhWm7vvAN4kuq6CmU0B8oH/ndBmV/3QnLtPB7LNrC/RWcnAhM0NAMqArUBPM8tqECexT1jfA6g4yLYa5vuwuxe7e3F+fnrnu4uIHImNOyqZt6aCCaP6EV1xSL9kzmrLN7OeYbkTcDGw3My+BVwKfN3d6xLaHxeuw2Bmo0Nu24APgKIwgy2HaHLAtHCNaDZwVdjEJOClsDwtvCesnxXaTwMmhllvg4EioutMIiLt0suLot+t/+q0fmnO5HNZzTc5bAXA1DBrLAN41t1fMbMYsBZ4P9SZF8IMtquAm8P6SmBiKBYxM7sVmAFkAo+6+5KwjzuAp83sp8CHwCMh/gjwOzMrJTrTmQjg7kvM7FlgKRADbnH3eBL/DURE0mrawjJGDezJ8X26pDuV/Sz62S5NKS4u9pKSknSnISJyyFZu3s2X7nubKX81ghvGDU7pvs1svrsXN7ZOdy4QEWmnpi0qI8Pgy6cWpDuVA6jwiIi0Q+7OtEVlnH1CX47plpfudA6gwiMi0g4tWLeDtdv2cUUrmlRQT4VHRKQdemreOrrkZHJ5KxtmAxUeEZF2Z2dlLa98VMYVo/rTNTeZk5cPjwqPiEg784cFG6iqreObZw1KdyqNUuEREWlH3J0n563j1AE9OLl/j3Sn0ygVHhGRdmT+2u18snkP3xjdOs92QIVHRKRdeXLuOrrmZrWqW+Q0pMIjItJO7NhXwysfb+LK0/vRpRVOKqinwiMi0k48NW89NbE6vnnW8elO5aBUeERE2oGq2jiPvvcp5xT1ZXhB93Snc1AqPCIi7cCLH26kfHc1N513QrpTaZYKj4hIGxevcx5+ezWn9O/B2Sf0SXc6zVLhERFp42Yu3czqrXv5+/OGtJqnjB6MCo+ISBvm7vz6rVUM6t2Z8SOPS3c6LZLMR1/nmdk8M1tkZkvM7MchPtjM5prZSjN7JjzOmvA46mfMrDSsL0zY1vdCfIWZXZoQHx9ipWZ2Z0L8kPchItIWzVldwcL1O/j2uUPIymwb5xLJzLIauNDdTwNGAePNbAzwM+A+dy8CtgM3hvY3AtvdfShwX2iHmY0genT1SGA88KCZZYZHav8KuAwYAXw9tOVQ9yEi0ha5O794fQXHdMvl6jMHpDudFkta4fHInvA2O7wcuBB4LsSnAleG5QnhPWH9RRYNVk4Annb3anf/FCgFRodXqbuvdvca4GlgQuhzqPsQEWlzZi3fQsna7dx2cRF52ZnpTqfFknpeFs5MFgJbgJnAKmCHu8dCkw1A/7DcH1gPENbvBPokxhv0aSre5zD2ISLSpsTrnJ//cQWFfTrzteKB6U7nkCS18Lh73N1HAQOIzlCGN9YsfG3szMOPYvxg+ziAmU02sxIzKykvL2+ki4hIek1btJEVm3dz+yUnkt1Gru3US0m27r4DeBMYA/Q0s/qbCA0AysLyBmAgQFjfA6hIjDfo01R862Hso2G+D7t7sbsX5+fnH95Bi4gkSU2sjl+8/gkj+3Xny6e0vieMNieZs9ryzaxnWO4EXAwsA2YDV4Vmk4CXwvK08J6wfpa7e4hPDDPSBgNFwDzgA6AozGDLIZqAMC30OdR9iIi0GU/OXcuG7ZX88/iTyMhoe5epk3n70gJgaph9lgE86+6vmNlS4Gkz+ynwIfBIaP8I8DszKyU6C5kI4O5LzOxZYCkQA25x9ziAmd0KzAAygUfdfUnY1h2Hsg8RkbZid1Ut988qZeyQPpxb1Dfd6RwW0y/8B1dcXOwlJSXpTkNEBIB7X1/B/bNKmXbrOE4d0DPd6TTJzOa7e3Fj69rWFSkRkQ5sy64qfvPOp3zl1IJWXXSao8IjItJG/OcbK6mN1/HdS05MdypHRIVHRKQNWFW+h2c+WM83zxpEYd8u6U7niKjwiIi0AffN/ITcrAz+8aKidKdyxFR4RERaudItu3n1401MOruQvl1z053OEVPhERFp5R6YVUqn7Ey+fc6QdKdyVKjwiIi0YqvK9/DyojKuHXs8vbvkpDudo0KFR0SkFfvVrFJys9rP2Q6o8IiItFqfbt3Liws38rdjBrWLazv1VHhERFqpB95YSXZmBpPPPSHdqRxVKjwiIq3Qis9284eFG7n+7ELyu7Wfsx1Q4RERaZX+4/UVdM3J4qbz2tfZDqjwiIi0OgvWbWfm0s1MPncIvdrJTLZEKjwiIq2Iu/PzPy6nb9cc/u6Lg9OdTlKo8IiItCLvrNzKnNUV3HLBULrkJvORaemjwiMi0krU1Tk/n7Gc/j078Y2zBqU7naRJ5qOvB5rZbDNbZmZLzOy2EH/GzBaG1xozWxjihWZWmbDu1wnbOtPMPjazUjO738wsxHub2UwzWxm+9gpxC+1KzewjMzsjYVuTQvuVZjYJEZFW4tWPN7F44y5uv2QYuVmZ6U4naZJ5HhcDbnf3BWbWDZhvZjPd/Zr6Bmb2C2BnQp9V7j6qkW09BEwG5gDTgfHAa8CdwBvufreZ3Rne3wFcBhSF11mh/1lm1huYAhQDHnKa5u7bj+aBi4gcqppYHf/x+gpOOq4bE0b1T3c6SdXiMx4z+6KZ3RCW883soFe93H2Tuy8Iy7uBZcD+f81w1vI14Klm9lsAdHf39z16TvfjwJVh9QRgalie2iD+uEfmAD3Ddi4FZrp7RSg2M4mKmIhIWj3zwTrWbtvHHeNPIjPD0p1OUrWo8JjZFKIzie+FUDbw+5buxMwKgdOBuQnhc4DN7r4yITbYzD40s7fM7JwQ6w9sSGizgc8L2LHuvgmiQgcck9BnfSN9moqLiKTN3uoY//XGSkYP7s35J+anO52ka+lQ21eJCkf9GUxZGD5rlpl1BZ4HvuPuuxJWfZ0Dz3Y2AYPcfZuZnQm8aGYjgcZKvze32yb6tGhbZjaZaGiPQYPa7wU+EWkdfvvOp2zdU8PD151EuITdrrV0qK0mDHM5gJm16LmrZpZNVHSecPcXEuJZwF8Dz9TH3L3a3beF5fnAKmAY0VnJgITNDgDKwvLmMIRWPyS3JcQ3AAMb6dNU/ADu/rC7F7t7cX5++//tQ0TSZ33FPh56q5TLTj6OMwb1Snc6KdHSwvOsmf030bWSbwN/An5zsA7hGs4jwDJ3v7fB6ouB5e6+IaF9vpllhuUhRBMDVochtN1mNiZs8zrgpdBtGlA/M21Sg/h1YXbbGGBn2M4M4BIz6xVmwF0SYiIiKefuTJm2hAwz/uUrI9KdTsq0aKjN3f/DzL4E7AJOBH7k7jOb6TYOuBb4uH7KNPB9d58OTOQvJxWcC9xlZjEgDtzk7hVh3c3AY0Anotlsr4X43URF8UZgHXB1iE8HLgdKgX3ADeE4KszsJ8AHod1dCfsQEUmpGUs2M2v5Fn745eH069kp3emkjEUjaM00iobWqtw9bmYnEhWf19y9NtkJpltxcbGXlJSkOw0RaWf2VMf40r1v0bNzDi/fOo6szPb19/xmNt/dixtb19IjfRvINbP+RMNsNxCdgYiIyGH4z5mf8NmuKv7tqye3u6LTnJYerbn7PqIJAQ+4+1eBjjMgKSJyFK3bto+p76/hmuKBHWZCQaIWFx4zGwt8E3g1xNrn3etERJLs3pkryMww/ulLw9KdSlq0tPDcRnQ7mhfcfUm4a8Gs5KUlItI+LS3bxUuLyrhh3GCO7Z6X7nTSoqVnLfuAOuDrZva3RH+I2fysBBEROcA9M5bTPS+7XT5ZtKVaWnieAL4LLCYqQCIicojmrt7G7BXl3HnZSfTolJ3udNKmpYWn3N1fTmomIiLt3D0zVnBs91yuP7sw3amkVUsLzxQz+y3wBlBdH0y8DY6IiDTtw3XbKVm7nX/9qxHkZbffZ+20REsLzw3ASUR3pa4fanNAhUdEpAX+5701dMvN4qrigc03budaWnhOc/dTkpqJiEg79dnOKqZ/vIlJZxfSNVd/idLS6dRzzEx/MCoichh+P2ctcXcmjS1MdyqtQktL7xeBSWb2KdE1HgPc3U9NWmYiIu1AVW2cJ+et4+LhxzKoT+d0p9MqtLTw6PHQIiKHYdrCMir21nDDuMJ0p9JqtPSxCGuTnYiISHvj7jz63qecdFw3xg7pk+50Wo2OdUtUEZEU+mDNdpZ/tpvrzy7sEI+0bikVHhGRJHn8/TV0z8tiwqj+6U6lVUla4TGzgWY228yWmdkSM7stxP/VzDaa2cLwujyhz/fMrNTMVpjZpQnx8SFWamZ3JsQHm9lcM1tpZs+YWU6I54b3pWF9YXP7EBE5mrbsquKPiz/j6uKBdMrp2H8w2lAyz3hiwO3uPhwYA9ySMCX7PncfFV7TAcK6icBIoskMD5pZppllAr8CLiN6BtDXE7bzs7CtImA7cGOI3whsd/ehwH2hXZP7SN4/gYh0VE/NW0+szvnbMcenO5VWJ2mFx903ufuCsLwbWAYc7HxzAvC0u1e7+6dAKTA6vErdfbW71wBPAxMsGjC9EHgu9J8KXJmwralh+TngotC+qX2IiBw1tfE6npy3lnOH5TO4b5d0p9PqpOQaTxjqOh2YG0K3mtlHZvaomdU/fq8/sD6h24YQayreB9jh7rEG8QO2FdbvDO2b2paIyFEzc+lmNu+q5jqd7TQq6YXHzLoCzwPfcfddwEPACcAoYBPwi/qmjXT3w4gfzrYa5jzZzErMrKS8vLyRLiIiTXv8/TX079mJC046Jt2ptEpJLTxmlk1UdJ6ov5O1u29297i71wG/4fOhrg1A4t3zBgBlB4lvBXqaWVaD+AHbCut7ABUH2dYB3P1hdy929+L8/PzDOXQR6aCWlO1kzuoKrh17PJkZmkLdmGTOajPgEWCZu9+bEC9IaPZVoofLAUwDJoYZaYOBImAe8AFQFGaw5RBNDpjm7g7MBq4K/ScBLyVsa1JYvgqYFdo3tQ8RkaPi12+tpmtuFl8fPSjdqbRaybxN6jjgWuBjM1sYYt8nmpU2imiIaw3w9wDuvsTMngWWEs2Iu8Xd4wBmdiswA8gEHnX3JWF7dwBPm9lPgQ+JCh3h6+/MrJToTGdic/sQETlS67bt49WPyvj2OUM69BNGm2PRiYA0pbi42EtKStKdhoi0Af/y4mKe+WA979xxAcd2z0t3OmllZvPdvbixdbpzgYjIUbB1TzXPlqznq6f37/BFpzkqPCIiR8HUP6+hJl7H5POGpDuVVk+FR0TkCO2uquXx99dy6YjjOCG/a7rTafVUeEREjtBv3vmUnZW13HLB0HSn0iao8IiIHIHy3dX89p3VfPmUAk4Z0CPd6bQJKjwiIkfggVkrqY7Vcfslw9KdSpuhwiMicpjWbtvLk3PXcc0XBjJE13ZaTIVHROQw/eL1T8jKNG67qCjdqbQpKjwiIodh0fodTFtUxt+NG6y/2zlEKjwiIocoXuf84MWPOaZbLjeff0K602lzVHhERA7R795fw+KNu/jRX42gW57uyXaoVHhERA7B5l1V/Mfrn3DusHy+fEpB8x3kL6jwiIgcgrteWUpNvI67rhhJ9PQXOVQqPCIiLfSnpZt59aNN3HrBUAr7dkl3Om2WCo+ISAuU767mjuc/YnhBd/5eNwI9Isl8EJyISLvg7vyf5xaxpzrGUxNHkZuVme6U2rRkPvp6oJnNNrNlZrbEzG4L8XvMbLmZfWRmfzCzniFeaGaVZrYwvH6dsK0zzexjMys1s/vDY7Uxs95mNtPMVoavvULcQrvSsJ8zErY1KbRfaWaTEBFpxuPvr+XNFeV8//LhDDu2W7rTafOSOdQWA2539+HAGOAWMxsBzAROdvdTgU+A7yX0WeXuo8LrpoT4Q8BkoCi8xof4ncAb7l4EvBHeA1yW0HZy6I+Z9QamAGcBo4Ep9cVKRKQxKz7bzb9PX8YFJ+Zz3djj051Ou5C0wuPum9x9QVjeDSwD+rv76+4eC83mAAMOth0zKwC6u/v7Hj2n+3HgyrB6AjA1LE9tEH/cI3OAnmE7lwIz3b3C3bcTFcH6IiYicoAd+2r49uMl9OiUzc+uOlWz2I6SlEwuMLNC4HRgboNVfwe8lvB+sJl9aGZvmdk5IdYf2JDQZkOIARzr7psgKnTAMQl91jfSp6m4iMgBYvE6bn3yQz7bWcWvrz2TY7rptjhHS9InF5hZV+B54Dvuvish/gOi4bgnQmgTMMjdt5nZmcCLZjYSaOxXDG9ut030adG2zGwy0RAdgwYNamZXItIe/fv05bxbupWfX3UqZwzSiPzRlNQzHjPLJio6T7j7CwnxScBXgG+G4TPcvdrdt4Xl+cAqYBjRWUnicNwAoCwsbw5DaPVDcltCfAMwsJE+TcUP4O4Pu3uxuxfn5+cfzqGLSBv25Nx1PPrep9wwrpCvFQ9svoMckmTOajPgEWCZu9+bEB8P3AFc4e77EuL5ZpYZlocQTQxYHYbQdpvZmLDN64CXQrdpQP3MtEkN4teF2W1jgJ1hOzOAS8ysV5hUcEmIiYgA8PKiMn7w4sdccGI+P7h8eLrTaZeSOdQ2DrgW+NjMFobY94H7gVxgZrhQNyfMYDsXuMvMYkAcuMndK0K/m4HHgE5E14TqrwvdDTxrZjcC64CrQ3w6cDlQCuwDbgBw9woz+wnwQWh3V8I+RKSDm718C//0zEK+UNibB795JlmZ+hv7ZLAw0iVNKC4u9pKSknSnISJJ9udVW7nhfz5g2LHdePLbZ+mu00fIzOa7e3Fj61TORaTDm7l0M9f/zwcc36czj93wBRWdJFPhEZEO7YUFG7jp9/MZXtCdZyaPpU/X3HSn1O7pXm0i0iG5Ow+/vZr/+9pyzj6hDw9fV0zXXP1ITAX9K4tIh1NVG+eO5z/ipYVlfPmUAn7xtdPIy9aNP1NFhUdEOpSNOyr5+9+VsKRsF9+9ZBi3XDBUt8JJMRUeEekwpi0q44d/+Jg6h99eV8xFw49Nd0odkgqPiLR7u6pqmfLSEv7w4UZGDezJf14zSk8QTSMVHhFpt+rqnOcWbOCeGSuo2FvDdy4u4tYLhuoPQ9NMhUdE2qU5q7fx01eXsnjjLs4Y1JNHJhVz6oCe6U5LUOERkXZmSdlO7pmxgjdXlFPQI4//mjiKK07rpwkErYgKj4i0C59u3ct9Mz9h2qIyenTK5s7LTmLS2EI65WiadGujwiMibVrZjkruf2Ml/2/+BnKzMrjlghOYfO4J9Oik2960Vio8ItImbdldxYOzV/Hk3HUAXDf2eP7h/KHkd9Mtb1o7FR4RaVN27qvlwbdKmfrnNdTGnavPHMA/XlRE/56d0p2atJAKj4i0CVW1cR5/fw2/mr2KXVW1TDitH9+5eJj+HqcNUuERkVatOhbn2ZINPDS7lLKdVZx/Yj7/fOlJjOjXPd2pyWFK5qOvB5rZbDNbZmZLzOy2EO9tZjPNbGX42ivEzczuN7NSM/vIzM5I2Nak0H6lmU1KiJ9pZh+HPveHR2Mf1j5EpHWJxeuY+uc1nPfzN/mXFxdT0LMTT37rLB67YbSKThuXzD/fjQG3u/twYAxwi5mNAO4E3nD3IuCN8B7gMqAovCYDD0FURIApwFnAaGBKfSEJbSYn9Bsf4oe0DxFpXcp2VPKN38xlyrQlHN+nM6718xcAABEOSURBVE9+6yyeu2ksZw/tm+7U5ChI2lCbu28CNoXl3Wa2DOgPTADOD82mAm8Cd4T44x49i3uOmfU0s4LQdqa7VwCY2UxgvJm9CXR39/dD/HHgSuC1Q91HyFVEWoE/Lv6MO57/iFi8jvuuOY2vnj4g3SnJUZaSazxmVgicDswFjq3/Qe/um8zsmNCsP7A+oduGEDtYfEMjcQ5jHyo8Imm2Y18Nd728lBc+3MipA3pw/8TTNXGgnUp64TGzrsDzwHfcfddBblvR2Ao/jPhB02lJHzObTDQUx6BBg5rZpIgcqT8u/owfvriYHftq+McLh/KPFxaRk6UbebZXSS08ZpZNVHSecPcXQnhz/fBWGErbEuIbgIEJ3QcAZSF+foP4myE+oJH2h7OPA7j7w8DDAMXFxc0VMxE5TOsr9vHjl5fyp2WbGdmvO1P/7guM7Ncj3WlJkiVzVpsBjwDL3P3ehFXTgPqZaZOAlxLi14WZZ2OAnWG4bAZwiZn1CpMKLgFmhHW7zWxM2Nd1DbZ1KPsQkRSqrInzn3/6hIvvfYs/r9rKnZedxIu3jFPR6SCSecYzDrgW+NjMFobY94G7gWfN7EZgHXB1WDcduBwoBfYBNwC4e4WZ/QT4ILS7q36iAXAz8BjQiWhSwWshfkj7EJHUqI7FeXreen45u5Ty3dV85dQCfvDl4RT00F0HOhKLJnhJU4qLi72kpCTdaYi0abXxOp6bv4EH3lhJ2c4qRg/uzXcvOZHRg3unOzVJEjOb7+7Fja3TnQtEJGli8Tpe+HAjD8xayfqKSkYN7MnPrzqNcUP76Pk4HZgKj4gcddWxOC8s2MiDb5ayvqKSUwf04K4JJ3P+sHwVHFHhEZGjZ091jGc/WM9v3lnNpp1VnDagB1O+MpKLhh+jgiP7qfCIyBHbsquKR99bwxNz17K7KsYXCntx99+cyrlFfVVw5C+o8IjIYavYW8Ov31oVno1Tx2UnF/CtcwZz+qBezXeWDkuFR0QOWWVNnN+8s5qH317N3poYXx3Vn9suLuL4PrrFjTRPhUdEWszdmbaojJ+9tpyynVVcOvJYbr/kRIYd2y3dqUkbosIjIi0yf+12/u3VpSxYt4OR/bpz3zWjOGtIn3SnJW2QCo+IHNT6in3c/cflvPrRJo7plsvP/+ZU/ubMAWRmaNKAHB4VHhFp1M7KWn41u5TH3ltDZoZx20VFTD53CF1y9WNDjoy+g0TkALXxOp6Ys5b/emMlOyprueqMAdx+yYkc1yMv3alJO6HCIyL7zV6xhZ++spRV5XsZN7QP3798uO4YLUedCo+IsL5iH//y0mLeXFHO4L5d+O11xbrbgCSNCo9IBxavc6b+eQ33zFhBhsEPvzyc68YW6umfklQqPCId1IfrtvPjl5eycP0Ozj8xn3/76in076nn4kjyqfCIdDBrtu7lnhkrePXjTfTtmst915zGlaP6a1hNUiZphcfMHgW+Amxx95ND7BngxNCkJ7DD3UeZWSGwDFgR1s1x95tCnzP5/Cmj04Hb3N3NrDfwDFAIrAG+5u7bw2Ow/4voSaP7gOvdfUHY1iTgh2EfP3X3qUk5eJFWaMVnu/nvt1cxbWEZOVkZfOfiIr59jqZHS+ol8zvuMeCXwOP1AXe/pn7ZzH4B7Exov8rdRzWynYeAycAcosIznugR13cCb7j73WZ2Z3h/B3AZUBReZ4X+Z4VCNQUoBhyYb2bT3H37UTlakVYoFq/j7ZXl/H7OOmYt30Kn7EyuHXs8N593Asd01/RoSY+kFR53fzucyfyFcFbyNeDCg23DzAqA7u7+fnj/OHAlUeGZAJwfmk4F3iQqPBOAxz16pvccM+sZtnM+MNPdK8K2ZhIVsacO9xhFWiN3Z+WWPUxbWMZz8zfw2a4q+nbN4fYvDePascfTs3NOulOUDi5d59jnAJvdfWVCbLCZfQjsAn7o7u8A/YENCW02hBjAse6+CcDdN5nZMSHeH1jfSJ+m4kkRr3Nq43VkZRiZGabxc0mqWLyOjzbuZNayLby2eBOryveSYXDesHz+9YroQWzZmZqpJq1DugrP1znwTGMTMMjdt4VrOi+a2UigsZ/W3sy2m+rT4m2Z2WSi4T0GDRrUzO4at3jjTib86r397+sLUHZmRvhqZGVkkJUZxerfZ2dlkB3aZWUaOZmft8nJzNgf398nxLIzjOysDLIyjJysEAtt6vvVx3OyjJzMTHKyolhu/Ss7k9zQRlo3d6d0yx7mfFrBn0u38l7pVnZVxcgwGDOkD9efXcilI4/TcJq0SikvPGaWBfw1cGZ9zN2rgeqwPN/MVgHDiM5KBiR0HwCUheXNZlYQznYKgC0hvgEY2EifDXw+NFcff7OxHN39YeBhgOLi4uYKXaOO65HHP48/kXjcqa1zYvE64nVOLCzXx2IJ62vjddTGnVhdHTWxOiprPcSidjX17Ru0rY0fVopNyswwcrMy6JSdSV52JrnZGeRlZZKXnUGnnEw6ZWeFrxl0zskiLzuTzjn1ryy65IavOZl0yc2iS24WXXOjeJecLDJ0c8lDVh2Ls7RsF/PXbqdkzXZK1lawdU8NAAU98hh/8nGcU5TPF4f2pVcXDaVJ65aOM56LgeXuvn8IzczygQp3j5vZEKKJAavdvcLMdpvZGGAucB3wQOg2DZgE3B2+vpQQv9XMniaaXLAzFKcZwL+bWf2jES8Bvpesgzy2ex7/cP7QZG3+AO4ehvac2ro6amN1+4tUTTwqYvXFqjoWFaqaWBSvicepro3iNbE6qmNxqmrrqKqNUxWLU1lTR1UsTnVtnMraaF3F3koqa2JU1dZRWRunsiZOTbyuxfnWF6RueVl0zcumW1iOXtl0z8ume6csuudl0zUvi265WXTNiwpYl5wsOocC1h7vjlwbr2N9xT5Kt+xhVfleVny2i2WbdrOqfA+xuugXjIG9O3FOUT5jhvRmzJA+DOrdWUO50qYkczr1U0RnGH3NbAMwxd0fASbylxf0zwXuMrMYEAduqp8EANzM59OpXwsviArOs2Z2I7AOuDrEpxNNpS4lmk59A0AoYj8BPgjt7krYR5tmZmRlGlmZ0InMtOQQi9exrzbO3uoY+2ri7KuOs6c6xt7qGHtrYuytjtbtqX9VRV93V8fYXVXLZ7uq2F1Vy67KGJW18RbtMy87gy45WfvPquoL2v6zq/3L9Wdd0ZlY19zojK1zTiads7PIzY6GGnOyouHMo3lNzt2pjtVRWRNnb02MXZUxduyrYUdlLeW7q9m8q4rNu6rZuGMf6ysq2bSzkrqEE9iCHnkML+jOxSOO4eR+PTjz+F4aPpM2z6LJX9KU4uJiLykpSXcaHUptvI49VTF2VdXuL1K7q6ICtq8mKmB7q+Psq4ntL257QmGLilxCoauJcajf4mbsv6aWVX8tLlyjy9w/WQQyzHD36EKhQ6zO908qqYnXhTPJ+AGFpKGsDCO/Wy79enZiYK9ODOzdmUG9OzP0mK6ccExXuudlH8k/pUjamNl8dy9ubJ3+ckxanezMDHp1yTkq1yrcncra6Oyr/ixsXzj7qKyJs68mfsAQY/21tuowVBmL11ETd+rC9bl4XR11DnXu1LljGOE/sjKiyR71Ezzqz6I652TtvwbWo1M2PTrl0KtLNvldc+nVOUfXvKTDUeGRds3Mwg/+LOiW7mxEBEDzZkVEJKVUeEREJKVUeEREJKVUeEREJKVUeEREJKVUeEREJKVUeEREJKVUeEREJKV0y5xmmFk5sDbdeRyhvsDWdCdxlOhYWqf2cizt5Tgg/cdyvLvnN7ZChacDMLOSpu6Z1NboWFqn9nIs7eU4oHUfi4baREQkpVR4REQkpVR4OoaH053AUaRjaZ3ay7G0l+OAVnwsusYjIiIppTMeERFJKRWedsTMxpvZCjMrNbM7G1mfa2bPhPVzzaww9Vm2TAuO5XozKzezheH1rXTk2Rwze9TMtpjZ4ibWm5ndH47zIzM7I9U5tlQLjuV8M9uZ8Jn8KNU5toSZDTSz2Wa2zMyWmNltjbRpE59LC4+l9X0u7q5XO3gBmcAqYAiQAywCRjRo8w/Ar8PyROCZdOd9BMdyPfDLdOfagmM5FzgDWNzE+suB14geYjoGmJvunI/gWM4HXkl3ni04jgLgjLDcDfikke+vNvG5tPBYWt3nojOe9mM0UOruq929BngamNCgzQRgalh+DrjIzFrjc5dbcixtgru/DVQcpMkE4HGPzAF6mllBarI7NC04ljbB3Te5+4KwvBtYBvRv0KxNfC4tPJZWR4Wn/egPrE94v4G//Abc38bdY8BOoE9Ksjs0LTkWgL8JwyDPmdnA1KR21LX0WNuKsWa2yMxeM7OR6U6mOWG4+XRgboNVbe5zOcixQCv7XFR42o/GzlwaTllsSZvWoCV5vgwUuvupwJ/4/EyurWkrn0lLLCC6TcppwAPAi2nO56DMrCvwPPAdd9/VcHUjXVrt59LMsbS6z0WFp/3YACT+1j8AKGuqjZllAT1onUMnzR6Lu29z9+rw9jfAmSnK7WhryefWJrj7LnffE5anA9lm1jfNaTXKzLKJflA/4e4vNNKkzXwuzR1La/xcVHjajw+AIjMbbGY5RJMHpjVoMw2YFJavAmZ5uPrYyjR7LA3G268gGttui6YB14VZVGOAne6+Kd1JHQ4zO67+mqGZjSb6+bItvVn9pZDjI8Ayd7+3iWZt4nNpybG0xs8lK507l6PH3WNmdiswg2hW2KPuvsTM7gJK3H0a0Tfo78yslOhMZ2L6Mm5aC4/lf5nZFUCM6FiuT1vCB2FmTxHNKuprZhuAKUA2gLv/GphONIOqFNgH3JCeTJvXgmO5CrjZzGJAJTCxlf5iMw64FvjYzBaG2PeBQdDmPpeWHEur+1x05wIREUkpDbWJiEhKqfCIiEhKqfCIiEhKqfCIiEhKqfCIiEhKqfCIJImZ7TnC/s+Z2ZBm2rxpZsVH2qZB+3wz+2NL24scKhUekVYo3E8r091Xp3rf7l4ObDKzcanet3QMKjwiSRb++v0eM1tsZh+b2TUhnmFmD4bnqLxiZtPN7KrQ7ZvASwnbeMjMSkLbHzexnz1m9gszW2Bmb5hZfsLqq81snpl9YmbnhPaFZvZOaL/AzM5OaP9iyEHkqFPhEUm+vwZGAacBFwP3hFv+/DVQCJwCfAsYm9BnHDA/4f0P3L0YOBU4z8xObWQ/XYAF7n4G8BbRnQXqZbn7aOA7CfEtwJdC+2uA+xPalwDnHPqhijRPt8wRSb4vAk+5exzYbGZvAV8I8f/n7nXAZ2Y2O6FPAVCe8P5rZjaZ6P/ZAmAE8FGD/dQBz4Tl3wOJN4ysX55PVOwgut3NL81sFBAHhiW03wL0O8TjFGkRFR6R5GvqYXsHewhfJZAHYGaDge8CX3D37Wb2WP26ZiTeD6v+Tt5xPv///p+AzURnYhlAVUL7vJCDyFGnoTaR5HsbuMbMMsN1l3OBecC7RA+zyzCzY4luwFlvGTA0LHcH9gI7Q7vLmthPBtENIQG+EbZ/MD2ATeGM61qiG7LWGwYsbsGxiRwynfGIJN8fiK7fLCI6C/lnd//MzJ4HLiL6Af8J0ZMjd4Y+rxIVoj+5+yIz+xBYAqwG3mtiP3uBkWY2P2znmmbyehB43syuBmaH/vUuCDmIHHW6O7VIGplZV3ffY2Z9iM6CxoWi1ImoGIwL14Zasq097t71KOX1NjDB3bcfje2JJNIZj0h6vWJmPYEc4Cfu/hmAu1ea2RSgP7AulQmF4cB7VXQkWXTGIyIiKaXJBSIiklIqPCIiklIqPCIiklIqPCIiklIqPCIiklIqPCIiklL/H9xjXuXs9YF/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "alpha is: 7.0435456241829275\n"
     ]
    }
   ],
   "source": [
    "mses = np.mean(lasso.mse_path_, axis = 1)\n",
    "plt.plot(np.log10(lasso.alphas_), mses) \n",
    "#plt.plot(np.log10(lasso.alphas_)*np.ones(3), [0.3, 0.4, 1.0])\n",
    "plt.xlabel('log(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show()    \n",
    "            \n",
    "print ('alpha is:', lasso.alpha_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef_lr</th>\n",
       "      <th>coef_ridge</th>\n",
       "      <th>coef_lasso</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>workingday</td>\n",
       "      <td>3.800799e+16</td>\n",
       "      <td>240.630984</td>\n",
       "      <td>143.965445</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>holiday</td>\n",
       "      <td>3.800799e+16</td>\n",
       "      <td>-345.294456</td>\n",
       "      <td>-235.180570</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_4</td>\n",
       "      <td>1.917491e+16</td>\n",
       "      <td>740.784787</td>\n",
       "      <td>447.825904</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_2</td>\n",
       "      <td>1.917491e+16</td>\n",
       "      <td>233.156303</td>\n",
       "      <td>58.655170</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_3</td>\n",
       "      <td>1.917491e+16</td>\n",
       "      <td>-315.111870</td>\n",
       "      <td>-351.417077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>season_1</td>\n",
       "      <td>1.917491e+16</td>\n",
       "      <td>-658.829219</td>\n",
       "      <td>-881.640800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>temp</td>\n",
       "      <td>4.984000e+03</td>\n",
       "      <td>2792.513136</td>\n",
       "      <td>4440.053403</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>atemp</td>\n",
       "      <td>5.540000e+02</td>\n",
       "      <td>2026.155271</td>\n",
       "      <td>297.992266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-1.706750e+03</td>\n",
       "      <td>-1412.035016</td>\n",
       "      <td>-1272.480306</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>hum</td>\n",
       "      <td>-2.898500e+03</td>\n",
       "      <td>-2309.435888</td>\n",
       "      <td>-1971.415870</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>-1.830830e+16</td>\n",
       "      <td>844.541521</td>\n",
       "      <td>408.410376</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>-1.830830e+16</td>\n",
       "      <td>460.963238</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-1.830830e+16</td>\n",
       "      <td>-1305.504759</td>\n",
       "      <td>-1653.088456</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>-5.589601e+16</td>\n",
       "      <td>162.522076</td>\n",
       "      <td>0.893561</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>-5.589601e+16</td>\n",
       "      <td>-57.858604</td>\n",
       "      <td>-98.918762</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>773.382980</td>\n",
       "      <td>657.756585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>422.254557</td>\n",
       "      <td>513.970956</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>220.040824</td>\n",
       "      <td>268.727878</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>112.652848</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-161.627991</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>59.058734</td>\n",
       "      <td>39.961395</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-130.514099</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-198.831620</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-198.153746</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-39.138426</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-279.213985</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-6.237891e+16</td>\n",
       "      <td>-579.910076</td>\n",
       "      <td>-568.652755</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "      <td>78.249508</td>\n",
       "      <td>12.324971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "      <td>32.423857</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "      <td>18.399871</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "      <td>-18.339437</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>-9.390400e+16</td>\n",
       "      <td>-215.397270</td>\n",
       "      <td>-189.409755</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns       coef_lr   coef_ridge   coef_lasso\n",
       "31    workingday  3.800799e+16   240.630984   143.965445\n",
       "30       holiday  3.800799e+16  -345.294456  -235.180570\n",
       "3       season_4  1.917491e+16   740.784787   447.825904\n",
       "1       season_2  1.917491e+16   233.156303    58.655170\n",
       "2       season_3  1.917491e+16  -315.111870  -351.417077\n",
       "0       season_1  1.917491e+16  -658.829219  -881.640800\n",
       "26          temp  4.984000e+03  2792.513136  4440.053403\n",
       "27         atemp  5.540000e+02  2026.155271   297.992266\n",
       "29     windspeed -1.706750e+03 -1412.035016 -1272.480306\n",
       "28           hum -2.898500e+03 -2309.435888 -1971.415870\n",
       "16  weathersit_1 -1.830830e+16   844.541521   408.410376\n",
       "17  weathersit_2 -1.830830e+16   460.963238    -0.000000\n",
       "18  weathersit_3 -1.830830e+16 -1305.504759 -1653.088456\n",
       "25     weekday_6 -5.589601e+16   162.522076     0.893561\n",
       "19     weekday_0 -5.589601e+16   -57.858604   -98.918762\n",
       "12        mnth_9 -6.237891e+16   773.382980   657.756585\n",
       "13       mnth_10 -6.237891e+16   422.254557   513.970956\n",
       "6         mnth_3 -6.237891e+16   220.040824   268.727878\n",
       "11        mnth_8 -6.237891e+16   112.652848     0.000000\n",
       "4         mnth_1 -6.237891e+16  -161.627991    -0.000000\n",
       "8         mnth_5 -6.237891e+16    59.058734    39.961395\n",
       "7         mnth_4 -6.237891e+16  -130.514099    -0.000000\n",
       "15       mnth_12 -6.237891e+16  -198.831620    -0.000000\n",
       "5         mnth_2 -6.237891e+16  -198.153746    -0.000000\n",
       "9         mnth_6 -6.237891e+16   -39.138426    -0.000000\n",
       "14       mnth_11 -6.237891e+16  -279.213985    -0.000000\n",
       "10        mnth_7 -6.237891e+16  -579.910076  -568.652755\n",
       "22     weekday_3 -9.390400e+16    78.249508    12.324971\n",
       "24     weekday_5 -9.390400e+16    32.423857     0.000000\n",
       "21     weekday_2 -9.390400e+16    18.399871     0.000000\n",
       "23     weekday_4 -9.390400e+16   -18.339437    -0.000000\n",
       "20     weekday_1 -9.390400e+16  -215.397270  -189.409755"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names1), \"coef_lr\":list((lr.coef_.T)), \"coef_ridge\":list((ridge.coef_.T)), \"coef_lasso\":list((lasso.coef_.T))})\n",
    "fs.sort_values(by=['coef_lr'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 再去掉特征\"temp\",\"season_4\",\"mnth_4\",\"weathersit_2\",\"weekday_2\"情况下再进行模型的训练看下结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 从原始数据中分离输入特征x和输出y\n",
    "y = df[\"cnt\"]\n",
    "\n",
    "X2 = df.drop([\"cnt\",\"instant\",\"yr\",\"temp\",\"season_4\",\"mnth_4\",\"weathersit_2\",\"weekday_2\"], axis = 1) \n",
    "#特征名称，用于后续显示权重系数对应的特征，在后面再实现\n",
    "feat_names2 = X2.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(584, 27)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#将数据分割训练数据与测试数据\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# 随机采样20%的数据构建测试样本，其余作为训练样本\n",
    "X2_train, X2_test, y_train, y_test = train_test_split(X2, y, random_state=33, test_size=0.2)\n",
    "X2_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 缺省参数的线性回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>workingday</td>\n",
       "      <td>1.820107e+17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>1.820107e+17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>holiday</td>\n",
       "      <td>1.820107e+17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>1.820107e+17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>atemp</td>\n",
       "      <td>4.750216e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>2.989119e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>2.487798e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>2.243891e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>9.346169e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>9.283011e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>8.209855e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>1.843760e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>-2.095820e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>-2.415052e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>-3.376975e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>-4.075142e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>-4.827059e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-5.167331e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>-5.363376e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_3</td>\n",
       "      <td>-6.614880e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_2</td>\n",
       "      <td>-7.356910e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-7.377327e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-1.270310e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-1.383459e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>season_1</td>\n",
       "      <td>-1.394206e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-1.413671e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>hum</td>\n",
       "      <td>-2.884003e+03</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns          coef\n",
       "26    workingday  1.820107e+17\n",
       "21     weekday_6  1.820107e+17\n",
       "25       holiday  1.820107e+17\n",
       "16     weekday_0  1.820107e+17\n",
       "22         atemp  4.750216e+03\n",
       "14  weathersit_1  2.989119e+02\n",
       "10        mnth_9  2.487798e+02\n",
       "19     weekday_4  2.243891e+02\n",
       "20     weekday_5  9.346169e+01\n",
       "6         mnth_5  9.283011e+01\n",
       "5         mnth_3  8.209855e+01\n",
       "11       mnth_10  1.843760e+01\n",
       "17     weekday_1 -2.095820e+02\n",
       "18     weekday_3 -2.415052e+02\n",
       "7         mnth_6 -3.376975e+02\n",
       "9         mnth_8 -4.075142e+02\n",
       "13       mnth_12 -4.827059e+02\n",
       "4         mnth_2 -5.167331e+02\n",
       "12       mnth_11 -5.363376e+02\n",
       "2       season_3 -6.614880e+02\n",
       "1       season_2 -7.356910e+02\n",
       "3         mnth_1 -7.377327e+02\n",
       "8         mnth_7 -1.270310e+03\n",
       "15  weathersit_3 -1.383459e+03\n",
       "0       season_1 -1.394206e+03\n",
       "24     windspeed -1.413671e+03\n",
       "23           hum -2.884003e+03"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 1.使用默认配置初始化学习器实例\n",
    "lr = LinearRegression()\n",
    "\n",
    "# 2.用训练数据训练模型参数\n",
    "lr.fit(X2_train, y_train)\n",
    "\n",
    "# 3. 用训练好的模型对测试集进行预测\n",
    "y_test_pred_lr = lr.predict(X2_test)\n",
    "y_train_pred_lr = lr.predict(X2_train)\n",
    "\n",
    "\n",
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names2), \"coef\":list((lr.coef_.T))})\n",
    "fs.sort_values(by=['coef'],ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of LinearRegression on test is 1290.6783904145111\n",
      "The RMSE of LinearRegression on train is 1264.8437142885846\n"
     ]
    }
   ],
   "source": [
    "# 使用RMSE评价模型在测试集和训练集上的性能，并输出评估结果\n",
    "\n",
    "#测试集,输出RMSE结果\n",
    "#print ('The r2 score of LinearRegression on test is', r2_score(y_test, y_test_pred_lr)) #r2_score评价\n",
    "print ('The RMSE of LinearRegression on test is', np.sqrt(mean_squared_error(y_test, y_test_pred_lr))) \n",
    "\n",
    "#训练集，输出RMSE结果\n",
    "#print ('The r2 score of LinearRegression on train is', r2_score(y_train, y_train_pred_lr)) #r2_score评价\n",
    "print ('The RMSE of LinearRegression on train is', np.sqrt(mean_squared_error(y_train, y_train_pred_lr))) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAesAAAFsCAYAAADsRpHBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAdZUlEQVR4nO3de5RdZZnn8e9DUiQ0RRICCYYETVBQwi3BApNBMBoVNEjoNSo4XMIQwAXqoI43ZJSI2i0N44WB1ma8AIISSGtj62I1TJp0hEEwhRGBgIlAxsQYQpBLUCSXZ/44O+GkciqpqlSl3qr6ftY66+z97n32ed9z6tTvvHu/e5/ITCRJUrl26+0KSJKk7TOsJUkqnGEtSVLhDGtJkgpnWEuSVDjDWpKkwhnWGvAi4uGImNbb9ehNEfG3EfH7iFgXEZN34fOui4gD21l2dkTc3U3P82REvL07tiX1BsNa/Vqjf9JtQyAzD83MBTvYzviIyIgY3ENV7W1XAh/OzObM/FXbhVXbX6zCdWVEfDUiBu3sk1bP9/jObkfq7wxrqQAFfAl4DfDwDtY5MjObgbcApwLn9HitJAGGtbRV7zsijomIRRHxfESsjoivVqstrO6frXqXUyNit4j4HxGxPCKeiogbImJ43XbPqpatjYjPtXmeORExLyJujIjngbOr5743Ip6NiFURcXVE7F63vYyICyNiaUS8EBFfjIjXVo95PiJuqV+/TRsb1jUihkTEOmAQ8OuI+N2OXq/MXAbcA0yq2/7wiPhOVe+VEfGlzT3viHhdRPxHRDwXEU9HxNw2bXpdNb1PRPykasv9wGvr1ttmz0ZELIiIc6vp10bEv1ev9dMRcVNEjGjntWjvPZaKZVhLW/sG8I3MHEYtLG6pyo+v7kdUu27vBc6ubm8FDgSagasBImIi8I/A6cAYYDgwts1zzQTmASOAm4CNwMeAfYGpwHTgwjaPORF4IzAF+BRwbfUcBwCHAR9op10N65qZf616y1DrOb+28cNfERFvAI4DltUVXw9sAF4HTAbeCZxbLfsicAewNzAO+F/tbPoa4CVqr9c5dK7nHsDfA/sDh1B7Pea0s25777FULMNaA8G/VL3VZyPiWWoh2p71wOsiYt/MXJeZv9jOuqcDX83MxzNzHXAxcFrV+3sv8K+ZeXdmvgx8Hmh7If57M/NfMnNTZv4lM1sz8xeZuSEznwT+idou53qXZ+bzmfkw8BBwR/X8zwG3UwvKzta1ox6IiBeBJcACqtcxIvYD3gV8NDNfzMyngK8Bp1WPW09tN/v+mflSZm4zaKzqhf9n4PPVNh6i9gWgQzJzWWbeWX35WAN8lW1fu8068x5LRTCsNRCckpkjNt/YtrdabzZwMPBoRPwyIk7azrr7A8vr5pcDg4H9qmW/37wgM/8MrG3z+N/Xz0TEwRHx04j4Y7Vr/O+o9bLrra6b/kuD+WYa215dO+qoavunAm8C9qzKXwM0AavqvhD9EzC6Wv4paj3f+6M28r5Rj3lUVZ/612R5g/UaiojREXFztQv+eeBGtn3tNuvMeywVwbCW6mTm0sz8ALWguRyYFxF7sm2vGOAP1IJqs1dT2xW8GlhFbZcvABGxB7BP26drM/9N4FHgoGoX7WephVx32F5dOyxrbgHupba3AGoB+1dg37ovRcMy89DqMX/MzPMyc3/gg8A/bj5OXWdNVZ8D2tRxsxer+7+pK3tV3fTfU3s9j6heuzNo57XbznssFcuwlupExBkRMSozNwHPVsUbqYXJJmrHezf7IfCxiJgQEc3UesJzM3MDtWPR74mI/1QN+voCOw7evYDngXXVceELuq1h269rV3wFOD8iXpWZq6gdk/6fETGsGsz22oh4C0BEvC8iNn9x+RO1UN1Yv7HM3Aj8CJgTEX9THfOfVbd8DbASOCMiBlW98/rj63sB66gNABwLfLK9im/nPZaKZVhLWzsReLgaIf0N4LTqOOufgS8D91S7eqcA3wW+T22k+BPUBkd9BKA6pvwR4GZqvewXgKeo9UDb8wngv1Tr/m9g7nbW7ax269oVmfkb4D94JRTPAnYHHqEWyPOoDRQDOBq4r3pNfwJclJlPNNjsh6ntZv8jcB3wvTbLz6ueby1wKPB/65Z9gdpu+ueAn1EL/vY0fI+332Kpd0Vmo717krpT1Zt9ltou7kZBJUntsmct9ZCIeE+1S3dPalcI+w3wZO/WSlJfZFhLPWcmtYFdfwAOora71V1ZkjrN3eCSJBXOnrUkSYXbpT8esO++++b48eN35VNKklSU1tbWpzNzVGces0vDevz48SxatGhXPqUkSUWJiA5fnW8zd4NLklQ4w1qSpMIZ1pIkFW6XHrOWJDW2fv16VqxYwUsveeXT/mLo0KGMGzeOpqamnd6WYS1JBVixYgV77bUX48ePJ6K7fmxNvSUzWbt2LStWrGDChAk7vT13g0tSAV566SX22Wcfg7qfiAj22WefbttTYlhLUiEM6v6lO99Pw1qSpMJ5zFqSCjRnzq7f3qBBgzj88MPZsGEDEyZM4Pvf/z4jRozo9HOde+65fPzjH2fixIlblV933XUsWrSIq6++utPbBGhubmbdunUdWnfatGlceeWVtLS0bClbtGgRN9xwA1dddVWXnr832bOWJAGwxx57sHjxYh566CFGjhzJNddc06XtfPvb394mqEvQ0tLS40G9cePGHtmuYS1J2sbUqVNZuXLllvkrrriCo48+miOOOIJLL70UgBdffJEZM2Zw5JFHcthhhzF37lyg1qvdfGnp733vexx88MG85S1v4Z577tmyvbPPPpt58+ZtmW9ubgZg3bp1TJ8+naOOOorDDz+c2267bZu6rVq1iuOPP55JkyZx2GGH8fOf/7xDbVqwYAEnnXQSAHPmzOGcc85h2rRpHHjggVuF+I033sgxxxzDpEmT+OAHP7glgC+44AJaWlo49NBDt7wGULuU9mWXXcab3/xmbr311g7VpbPcDS5J2srGjRuZP38+s2fPBuCOO+5g6dKl3H///WQmJ598MgsXLmTNmjXsv//+/OxnPwPgueee22o7q1at4tJLL6W1tZXhw4fz1re+lcmTJ2/3uYcOHcqPf/xjhg0bxtNPP82UKVM4+eSTtxqs9YMf/IATTjiBSy65hI0bN/LnP/+5S+189NFHueuuu3jhhRd4/etfzwUXXMCyZcuYO3cu99xzD01NTVx44YXcdNNNnHXWWXz5y19m5MiRbNy4kenTp/Pggw9yxBFHbKn33Xff3aV6dIRhLUkC4C9/+QuTJk3iySef5I1vfCPveMc7gFpY33HHHVuCdt26dSxdupTjjjuOT3ziE3z605/mpJNO4rjjjttqe/fddx/Tpk1j1KjaD0ydeuqp/Pa3v91uHTKTz372syxcuJDddtuNlStXsnr1al71qldtWefoo4/mnHPOYf369ZxyyilMmjSpS+2dMWMGQ4YMYciQIYwePZrVq1czf/58WltbOfroo7e8JqNHjwbglltu4dprr2XDhg2sWrWKRx55ZEtYn3rqqV2qQ0e5G1ySBLxyzHr58uW8/PLLW45ZZyYXX3wxixcvZvHixSxbtozZs2dz8MEH09rayuGHH87FF1/MZZddts022zt9afDgwWzatGnL9l9++WUAbrrpJtasWUNrayuLFy9mv/322+Zc5eOPP56FCxcyduxYzjzzTG644YYutXfIkCFbpgcNGsSGDRvITGbNmrWlrY899hhz5szhiSee4Morr2T+/Pk8+OCDzJgxY6t67bnnnl2qQ0fZs1a/09FRtN092lbqL4YPH85VV13FzJkzueCCCzjhhBP43Oc+x+mnn05zczMrV66kqamJDRs2MHLkSM444wyam5u57rrrttrOm970Ji666CLWrl3LsGHDuPXWWznyyCOB2nHe1tZW3v/+93Pbbbexfv16oLYrffTo0TQ1NXHXXXexfPm2vya5fPlyxo4dy3nnnceLL77IAw88wFlnndUtbZ8+fTozZ87kYx/7GKNHj+aZZ57hhRde4Pnnn2fPPfdk+PDhrF69mttvv51p06Z1y3N2hGEtSQXq7S+TkydP5sgjj+Tmm2/mzDPPZMmSJUydOhWoDQa78cYbWbZsGZ/85CfZbbfdaGpq4pvf/OZW2xgzZgxz5sxh6tSpjBkzhqOOOmrLYK3zzjuPmTNncswxxzB9+vQtPdPTTz+d97znPbS0tDBp0iTe8IY3bFO3BQsWcMUVV9DU1ERzc3O7PesZM2ZsuS731KlT+dCHPrTDdk+cOJEvfelLvPOd72TTpk00NTVxzTXXMGXKFCZPnsyhhx7KgQceyLHHHtvxF7MbRGbusidraWnJzSMEpZ5iz1p90ZIlSzjkkEN6uxrqZo3e14hozcyWdh7SkMesJUkqnGEtSVLhDGtJKsSuPCypnted76dhLUkFGDp0KGvXrjWw+4nNv2c9dOjQbtmeo8ElqQDjxo1jxYoVrFmzprerom4ydOhQxo0b1y3bMqwlqQBNTU1MmDCht6uhQrkbXJKkwhnWkiQVzrCWJKlwhrUkSYUzrCVJKpxhLUlS4QxrSZIKZ1hLklQ4w1qSpMIZ1pIkFc6wliSpcIa1JEmFM6wlSSqcYS1JUuEMa0mSCmdYS5JUOMNakqTCGdaSJBXOsJYkqXCGtSRJhTOsJUkqXIfDOiIGRcSvIuKn1fyEiLgvIpZGxNyI2L3nqilJ0sDVmZ71RcCSuvnLga9l5kHAn4DZ3VkxSZJU06GwjohxwAzg29V8AG8D5lWrXA+c0hMVlCRpoOtoz/rrwKeATdX8PsCzmbmhml8BjG30wIg4PyIWRcSiNWvW7FRlJUkaiHYY1hFxEvBUZrbWFzdYNRs9PjOvzcyWzGwZNWpUF6spSdLANbgD6xwLnBwR7waGAsOo9bRHRMTgqnc9DvhDz1VTkqSBa4c968y8ODPHZeZ44DTg3zPzdOAu4L3VarOA23qslpIkDWA7c571p4GPR8Qyasewv9M9VZIkSfU6sht8i8xcACyoph8Hjun+KkmSpHpewUySpMIZ1pIkFc6wliSpcIa1JEmFM6wlSSqcYS1JUuEMa0mSCmdYS5JUuE5dFEXqT+bM6RvblCR71pIkFc6wliSpcIa1JEmFM6wlSSqcYS1JUuEMa0mSCmdYS5JUOMNakqTCGdaSJBXOsJYkqXCGtSRJhTOsJUkqnGEtSVLhDGtJkgpnWEuSVDjDWpKkwhnWkiQVzrCWJKlwhrUkSYUb3NsVkDQwzJnTM+tKA4E9a0mSCmdYS5JUOMNakqTCGdaSJBXOsJYkqXCGtSRJhTOsJUkqnGEtSVLhDGtJkgpnWEuSVDjDWpKkwhnWkiQVzrCWJKlwhrUkSYUzrCVJKpxhLUlS4QxrSZIKZ1hLklQ4w1qSpMIZ1pIkFc6wliSpcIa1JEmFM6wlSSqcYS1JUuEMa0mSCmdYS5JUuMG9XQGpo+bM6e0aSFLvsGctSVLhDGtJkgq3w7COiKERcX9E/DoiHo6IL1TlEyLivohYGhFzI2L3nq+uJEkDT0d61n8F3paZRwKTgBMjYgpwOfC1zDwI+BMwu+eqKUnSwLXDsM6addVsU3VL4G3AvKr8euCUHqmhJEkDXIdGg0fEIKAVeB1wDfA74NnM3FCtsgIY285jzwfOB3j1q1+9s/WVNAB0dOS/ZwhooOjQALPM3JiZk4BxwDHAIY1Wa+ex12ZmS2a2jBo1qus1lSRpgOrUaPDMfBZYAEwBRkTE5p75OOAP3Vs1SZIEHRsNPioiRlTTewBvB5YAdwHvrVabBdzWU5WUJGkg68gx6zHA9dVx692AWzLzpxHxCHBzRHwJ+BXwnR6spyRJA9YOwzozHwQmNyh/nNrxa0mS1IO8gpkkSYXzhzwk9XueCqa+zp61JEmFM6wlSSqcYS1JUuEMa0mSCmdYS5JUOMNakqTCeeqW1Av606lEvVnHvvD6SN3BnrUkSYUzrCVJKpxhLUlS4QxrSZIKZ1hLklQ4R4NL3cjRyZJ6gj1rSZIKZ1hLklQ4w1qSpMIZ1pIkFc6wliSpcIa1JEmFM6wlSSqcYS1JUuEMa0mSCmdYS5JUOMNakqTCGdaSJBXOsJYkqXCGtSRJhTOsJUkqnGEtSVLhDGtJkgpnWEuSVDjDWpKkwhnWkiQVzrCWJKlwhrUkSYUzrCVJKpxhLUlS4QxrSZIKZ1hLklQ4w1qSpMIZ1pIkFc6wliSpcIa1JEmFM6wlSSqcYS1JUuEMa0mSCmdYS5JUOMNakqTCGdaSJBXOsJYkqXCGtSRJhTOsJUkqnGEtSVLhBvd2BSSpFHPm9I1tauCxZy1JUuEMa0mSCrfDsI6IAyLirohYEhEPR8RFVfnIiLgzIpZW93v3fHUlSRp4OtKz3gD898w8BJgCfCgiJgKfAeZn5kHA/GpekiR1sx2GdWauyswHqukXgCXAWGAmcH212vXAKT1VSUmSBrJOHbOOiPHAZOA+YL/MXAW1QAdGt/OY8yNiUUQsWrNmzc7VVpKkAajDYR0RzcA/Ax/NzOc7+rjMvDYzWzKzZdSoUV2poyRJA1qHwjoimqgF9U2Z+aOqeHVEjKmWjwGe6pkqSpI0sHVkNHgA3wGWZOZX6xb9BJhVTc8Cbuv+6kmSpI5cwexY4EzgNxGxuCr7LPAV4JaImA38P+B9PVNFSZIGth2GdWbeDUQ7i6d3b3UkSVJbXsFMkqTC+UMe6lX+yIEk7Zg9a0mSCmdYS5JUOMNakqTCGdaSJBXOsJYkqXCGtSRJhfPULalgHT21zVPgpP7NnrUkSYUzrCVJKpxhLUlS4QxrSZIKZ1hLklQ4w1qSpMIZ1pIkFc6wliSpcIa1JEmFM6wlSSqcYS1JUuEMa0mSCucPeahH+MMSktR97FlLklQ4w1qSpMIZ1pIkFc6wliSpcIa1JEmFM6wlSSqcp25JA4yn1Ul9jz1rSZIKZ1hLklQ4w1qSpMIZ1pIkFc6wliSpcIa1JEmF89QtSepBHT1VzlPqtD32rCVJKpxhLUlS4QxrSZIKZ1hLklQ4w1qSpMI5Glyd4ohVSdr17FlLklQ4w1qSpMIZ1pIkFc6wliSpcIa1JEmFM6wlSSqcp25J/YCn1En9mz1rSZIKZ1hLklQ4w1qSpMIZ1pIkFc6wliSpcIa1JEmF89QtedqPJBXOnrUkSYUzrCVJKtwOwzoivhsRT0XEQ3VlIyPizohYWt3v3bPVlCRp4OpIz/o64MQ2ZZ8B5mfmQcD8al6SJPWAHYZ1Zi4EnmlTPBO4vpq+Hjilm+slSZIqXT1mvV9mrgKo7kd3X5UkSVK9Hh9gFhHnR8SiiFi0Zs2ann46SZL6na6G9eqIGANQ3T/V3oqZeW1mtmRmy6hRo7r4dJIkDVxdDeufALOq6VnAbd1THUmS1FZHTt36IXAv8PqIWBERs4GvAO+IiKXAO6p5SZLUA3Z4udHM/EA7i6Z3c10kSVIDXsFMkqTC+UMeklSAzvygjj++M/DYs5YkqXCGtSRJhTOsJUkqnGEtSVLhDGtJkgpnWEuSVDhP3ZKkPqajp255ilf/Yc9akqTCGdaSJBXOsJYkqXCGtSRJhTOsJUkqnGEtSVLhPHVLkvopT/HqP+xZS5JUOMNakqTCGdaSJBXOsJYkqXCGtSRJhTOsJUkqnGEtSVLhDGtJkgpnWEuSVDjDWpKkwhnWkiQVzrCWJKlw/pBHP+bF+SWpf7BnLUlS4QxrSZIKZ1hLklQ4w1qSpMIZ1pIkFc6wliSpcJ661UZHT3fytChJ/YX/98pnz1qSpMIZ1pIkFc6wliSpcIa1JEmFM6wlSSqcYS1JUuE8dasP8vQJSRpY7FlLklQ4w1qSpMIZ1pIkFc6wliSpcIa1JEmF69OjwTszKrovjKDuC3WUNHD1xP+o3vy/15d+wMSetSRJhTOsJUkqnGEtSVLhDGtJkgpnWEuSVDjDWpKkwvXpU7d6UwlD+SVJA4M9a0mSCmdYS5JUOMNakqTC7VRYR8SJEfFYRCyLiM90V6UkSdIruhzWETEIuAZ4FzAR+EBETOyuikmSpJqd6VkfAyzLzMcz82XgZmBm91RLkiRtFpnZtQdGvBc4MTPPrebPBN6UmR9us975wPnV7OuBx7pe3V61L/B0b1eih9nG/qG/t7G/tw9sY3/RXhtfk5mjOrOhnTnPOhqUbZP8mXktcO1OPE8RImJRZrb0dj16km3sH/p7G/t7+8A29hfd2cad2Q2+Ajigbn4c8Iedq44kSWprZ8L6l8BBETEhInYHTgN+0j3VkiRJm3V5N3hmboiIDwP/BgwCvpuZD3dbzcrT53fld4Bt7B/6exv7e/vANvYX3dbGLg8wkyRJu4ZXMJMkqXCGtSRJhTOsKxHxxYh4MCIWR8QdEbF/VR4RcVV1SdUHI+KousfMioil1W1WXfkbI+I31WOuiohGp7ntUhFxRUQ8WrXhxxExom7ZxVVdH4uIE+rKG15OthpUeF/V7rnVAMNeFxHvi4iHI2JTRLS0WdYv2rg9ffnyvxHx3Yh4KiIeqisbGRF3Vu/BnRGxd1Xe6c9kb4uIAyLirohYUv2NXlSV96c2Do2I+yPi11Ubv1CVN/wsRcSQan5ZtXx83bYafl5LERGDIuJXEfHTar7n25iZ3mrH7YfVTf834FvV9LuB26mdVz4FuK8qHwk8Xt3vXU3vXS27H5haPeZ24F0FtO+dwOBq+nLg8mp6IvBrYAgwAfgdtQGDg6rpA4Hdq3UmVo+5BTitmv4WcEFvt6+qyyHULryzAGipK+83bdxO29ttS1+4AccDRwEP1ZX9A/CZavozdX+znf5M9vYNGAMcVU3vBfy2+rvsT20MoLmabgLuq+re8LMEXMgr/2dPA+ZW0w0/r73dvjZt/TjwA+Cn1XyPt9GedSUzn6+b3ZNXLvAyE7gha34BjIiIMcAJwJ2Z+Uxm/gm4EzixWjYsM+/N2rtyA3DKrmtJY5l5R2ZuqGZ/Qe28eKi17+bM/GtmPgEso3Yp2YaXk632ErwNmFc9/noKaB9AZi7JzEZXyOs3bdyOPn3538xcCDzTpngmtdcetn4POvWZ7Pna71hmrsrMB6rpF4AlwFj6VxszM9dVs03VLWn/s1Tf9nnA9Oqz197ntQgRMQ6YAXy7mt/e/4tua6NhXScivhwRvwdOBz5fFY8Ffl+32oqqbHvlKxqUl+Qcat/aofPt2wd4ti74S2xfWwO5jX3Zfpm5CmphB4yuyjv7fhal2hU6mVrPs1+1sdo9vBh4itoXid/R/mdpS1uq5c9R++wV3Ubg68CngE3V/Pb+X3RbGwdUWEfE/4mIhxrcZgJk5iWZeQBwE7D5GuftXVa1s+U9bkftq9a5BNhArY1sp77FtQ861sZGD2tQVmwbu6gv1rmr+uz7FhHNwD8DH22zN2+bVRuUFd/GzNyYmZOo7bk7htqhqW1Wq+77XBsj4iTgqcxsrS9usGq3t3Fnrg3e52Tm2zu46g+AnwGX0v5lVVcA09qUL6jKxzVYv8ftqH3VYJSTgOnVLnrY/mVjG5U/TW2X3ODqm+IuvcxsJ97Den2qjV3UHy//uzoixmTmqmoX8FNVeWc/k0WIiCZqQX1TZv6oKu5XbdwsM5+NiAXUjlm391na3MYVETEYGE7tUEjJf8vHAidHxLuBocAwaj3tnm9jbx+oL+UGHFQ3/RFgXjU9g60HetxflY8EnqA2yGPvanpkteyX1bqbB5i9u4D2nQg8AoxqU34oWw90eJzaYKXB1fQEXhmwdGj1mFvZejDFhb3dvjZtWsDWA8z6XRsbtLndtvSVGzCerQeYXcHWg6/+oZru9Geyt29VXW8Avt6mvD+1cRQwopreA/g5tc5Bw88S8CG2Hnx1SzXd8PPa2+1r0N5pvDLArMfb2OsNLuVG7RvvQ8CDwL8CY6vyAK6hduzlN2wdAudQGxiwDPivdeUt1bZ+B1xNdaW4Xm7fMmrHSBZXt2/VLbukqutj1I1cpzYi9bfVskvqyg+kNuJ9WfVHOqS321fV62+pfWP9K7Aa+Lf+1sYdtL9hW/rCDfghsApYX72Hs6kd25sPLK3uN38Z7vRnsrdvwJup7eZ8sO4z+O5+1sYjgF9VbXwI+HxV3vCzRK1nemtVfj9wYN22Gn5eS7qxdVj3eBu93KgkSYUbUAPMJEnqiwxrSZIKZ1hLklQ4w1qSpMIZ1pIkFc6wliSpcIa1JEmF+/8460yva7pWrwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#在训练集上观察预测残差的分布，看是否符合模型假设：噪声为0均值的高斯噪声\n",
    "f, ax = plt.subplots(figsize=(7, 5)) \n",
    "f.tight_layout() \n",
    "ax.hist(y_train - y_train_pred_lr, bins=40, label='Residuals Linear', color='b', alpha=.5); \n",
    "ax.set_title(\"Histogram of Residuals\") \n",
    "ax.legend(loc='best');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAADQCAYAAADcQn7hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO2de5QcdZ3oP9/pdJJOQCaRoGFISEBuOERMAlkJ5t4VUInyjIoGFlz0rrA+VgE5kbByNokv4sYVdO9elSvLQWEh4eEYRA1I4O5d1iAJk4ARsoSHIQNKNJmoyUA6M9/7R1V1amrq1Y/q13w/5/Tp7qrqqm9Xd33r9/s+RVUxDMPIgo5GC2AYRvtiCsYwjMwwBWMYRmaYgjEMIzNMwRiGkRmjGi1AFhx++OE6bdq0RothGCOGjRs3/l5VJwWXt6WCmTZtGhs2bGi0GIYxYhCR34QttymSYRiZYQrGMIzMMAVjGEZmtKUNxjDC6O7pZeXarbzc18+RnQUWL5jBwjldjRarrTEFY4wIunt6ufbep+gvDgDQ29fPtfc+BWBKJkNsimSMCFau3VpSLh79xQFWrt3aIIlGBjaCMUYEL/f1p15uU6naYSMYY0RwZGch1XJvKtXb149ycCrV3dNbBynbD1Mwxohg8YIZFPK5IcsK+RyLF8wYssymUrXFpkjGiMCb4iRNfcqZShnJmIIxMqdZbBoL53QlHvfIzgK9IcokaoplxGNTJCNTWs2mkXYqZaTDRjBGpsTZNJrRM5N2KtVKNHIEaQrGyJSsbBpZXjRpplKtQqMDDE3BGJlSS5uGp1R6+/oRwOuH0Y5RubVSoI0eQZqCMTJl8YIZQ+6gUJlNI3gnDjbb6S8OcPXqzUC8kom7cCtdV2tqOepotFfMFIyRKbWyaYTdiYMMqHLVqk1cuWoTXSHHibtwgYrWZaFkajnqaLRXTNqx8drcuXPVKtrVnnobC/3Hq+RfWsjnuP4DJ5ZknL9iXejF1uVebJWse3TJGRVIFs/0JfdHft9L5k3l4Wd2DvkNIFqBB5UqDD8vtUBENqrq3OByG8EYqai3sbC7p5fFd22mOFj5DTA4bapkulDpukrp7umlQ4SBiBv/beu3l1739vXzuVWbGPStD/4ujfaKmYIxUlFvY+GyNVuqUi4eA6qlCy5pulDpulrhKfEo5RLGYMiyoGJN4xXLanRqgXZGIt09vaEXGGRnLOzrL9ZsX54ijAuiK3ed4Cid+SvW1SxoMI2dKS2eYk0jW5bBkDaCMWLx/nxRNFMIvd91HeTlvv5U04WkdZ6irdRFfl33U9zx2EsMqJIT4aJTpvDlhSeWZKwlaT1rWY5OTcEYscTdVasNoY8blk8Yl2f3vnSjmHyHsPJDswC4evXm0CmGpwjjpgtJ6wCuWrUp1EUedjEGv9+0NxZ49LldpfUDqty2fjsv7PwzL/6hMkN2Ev4pYtR3y9KVbQrGiCXuTxbniUia04cZjRfftZnl922hb1+RznH59EKK8+Ttv5y4m3JsDyvXbo0dIQX3G/x+UdNMv9LJgqTRSJRtqkOE6Uvur8omYzYYI5aoKVBXZyFWuSTN6cNGRsVBZfe+IgqpRy8AxQEt1WtZOKeL6z9wYsm9nBMpXWBBm0K5toc4ZRs8T7W0p9SCONnDbEzgjH6qtcmYgjFiqSS7OE3RplrbG4IG172vHwAoTZd6+/q5atUmrus+aE8qt7hUlLIVGHY+GlE/Zv6xE8mJhK7rEIlUEH6lLBC6j0qLbmWqYESkU0TuFpFnRORpETlVRCaKyIMi8qz7PMHdVkTkWyKyTUSeFJGTfPu51N3+WRG5NEuZjaEE/3xdnYXEIK00c/osjMO9ff0svnszi+/aHOqFUuD29dtLF1q5tofFC2aQ7xh+8V08b+qw89EI4/d/PreLAVXCVMyAKleu2sTs5Q+EKpqFc7p4dMkZvLDibAYj3OSVKM2sRzDfBH6mqscDs4CngSXAQ6p6HPCQ+x7gfcBx7uNy4NsAIjIRWAqcArwdWOopJaM++P98jy45I1XRpqTlUcPyaikOaGz8jELpTpy2Tu8QAldvPifMPXpi6X13T28pYjh8LJEdGngOo6+/mDjdqei8RJCZghGRNwB/CdwMoKr7VbUPOB+41d3sVmCh+/p84PvqsB7oFJHJwALgQVXdpaq7gQeB92Ylt1E9aaZVwZFRZ6EMo26V9Pb1M33J/ezbf2DYiESA04+fFPq5lWu3UhwYevn67T9+mw7EX+iNJGm6U8uiW1l6kY4BdgK3iMgsYCNwBfAmVX0FQFVfEZEj3O27gJd8n9/hLotaPgQRuRxn5MPUqVNr+02MSOK8MEneGb9buLunlytXbaqb3J4hORdQMIoTjn//k6+w9NyZQ2ROmlI1m2E3jiiPFtS26FaWCmYUcBLwGVV9TES+ycHpUBhhI0qNWT50gepNwE3gJDuWL65RLkn5SWF/yDCFBMQG82XJQMR0avc+Zyqx4Te7SsmFUTlC3tSh1QqDz17+AHv6i6EKpFZFt7K0wewAdqjqY+77u3EUzu/cqQ/u86u+7af4Pn8U8HLMcqPBlOuFiXILL79vS1Pe+fuLA9y+fntJ3jDl4p86HFbHaV4t6OsvZl4nOTMFo6q/BV4SEW/i9i7g18AawPMEXQr8yH29Bvhr15s0D9jjTqXWAmeKyATXuHumu8yoA57RcvqS+4fl3UTdsT0bR3D7KIVUTsxLvQkb3+REQj1qxYGw1MPWIKveT1lH8n4GuF1ERgPPAx/DUWqrReRvgO3Ah9xtfwKcBWwD9rnboqq7RORLwOPudl9U1WxDHw0geQoUFQEKDLkzets34xQi1yF0QFmZ2wOq3LhoNgvndNHd08vs5Q/UNDmzUWTx+1jBqRFOnJE2rkDTo0vOCC1mFIa3fdT+GklnIc85syYPqbPSbkwYl+e14mDq36kSogpOWSTvCCYpVD7Ja7JwThcfPLkrMno0uH1WsS/V0Ndf5J6NzdmjqRYU8jmWnjtzSEjAhHH5YRd+vkMy6f1kyY4jmKQ0/aQCTd09vdyzsTexQJKXuOh3f9ZrJHPcEeN5fue+SBm9XKV2xW8j8ocELL57M4P+mJ6MogJtBNNmxBllgySNUJICrtLGffiv7YVzuli8YAYTysmWrpD5x07kwc+dFhn6DuGeIT/1jsatJV7CZ/D/kBQwWEtMwbQR5WYHJ4WEh+UhffDkLlau3cr0JfenHoXs8RlAvbtn1p6j8aNzfGiuE3AZ9T07C/nSRRhGIZ/j4nlTG6JkclUetJDPcfrxk4b9H65atamu1QlNwbQR5calRI1QTj9+Uumu55WafGHF2SxeMIN7NvaW/rBp8V/gy+/bMuzumQV79w+w+O7NjkKL+J7LzpsZaRcal+/g+g+cyJcXnsjF8+Ijw/M1uIo8O1ZOhPnHTgyttZtEMCH14Wd2Dvs/xJ35LBI0zQbTRpSbHRwWEn768ZO4Z2NvqGu6klD4fE7Y+/qBUuGiesa8eMN+zzMS/J7e+0KIhvBykj07UxyHjM0PSSuYtuT+smUdm+/gK+93Smdee+9TlOvcDZtyxo1IguVFq61OGHkcc1O3D0lu5Wr3kdSfqJDv4KSpnax/fjcDqnS4/+JGhp8J8MKKs4csS+teB2dEkabKv3fBdnUW2Pv6gYriYnIdwqFjRpX92XxOQIfG8hTyOcbmO2IVuveb1qKLgPVFGgHUok1r3CgoLrAOoL84yC+e21VSKDXoOlI1ijOi6CzkWXaeM8ooZySWtoWIvwh4PifkO6TstisDg5qoXMaPzjGoSn/ROcsTxuVRHd6Fob84wJhRHZGF0LNqGhfEbDBtRCXFoYLEGX7TxLE0a7B8X3+RK92KdllHFBcHlEPGjoqtEFcuOREumTeVQaWkXABeKw5GKqU9/cVQI3VW06EwbIpkDCGp1ajn5my2iNxyGJfvYF8xW1Xon5rFtYINEhZ1641CoqZrUcv9EddZd3aseIokIm8CvgocqarvE5ETgFNV9eaaSmjUlag/XVQtEHDsM96yDmmOKVAlVKJc0tpiPLyp2YRxeTrLaMGiCh88uYuHn9lZqornHTXq+AOqFPK5yKlxrUovVELiCEZEfgrcAnxBVWeJyCigR1VPrIeAldBKI5h6N5T3jllOQ/Sw7Tto3ulQrRGcurt+71q55DoksvZMEO+3SDtS7HL/N43qPw3VGXkPV9XVInItgKoeEJH2ja2uI/VuKO9Rbie/sO0HcbxG+w9oWXf2VsNTLmExJeUwMKilZnLeaCjK4+R1ZExzXr2RSiNHKXGkUTB7ReSNuCM1r1ZLplKNELJq2Rk3KkrqM+3/bGeEh+KgrIO8uOLsupYr6Czk63asnAjHTBpXavVaLeNGj6LnH84csmx6RMxM3PFyIgyqNmSkUi5ppkgnAf8MvBX4FTAJuEBVn8xevMpolSlSlPEvLHYjLXHTHxje9dBP2rT+RuE3Wi6+e3NdIoJrjRd7clghj0h5DeYgfirbSCqeIqnqEyLyTmAGzn9/q6q2fnWdJiApW7kSktIF4vpMq0avbwZe2dPPtCX309VZYNFfTCnVykUIjXyNigFpFMLBYtvljMJaacQSJDEORkQ+DRyiqltU9VfAISLyqexFa39q2R7CIy5QLqnP9J46TT0mjMtzybyphPQwi8Wzkfb29XPPRifH6OJ5U0OVSwfNpVygcnkGVVP3pGo20gTaXeb2MwLA7U10WXYijRxqERgXJC5QLm5ktHLt1roVrd69r8ht67dX5ebuLw6w/L4t3B5Ria6ZPFzVhtk1oktkrUhj5O0QEVHXWCMiOWB0tmKNHGpt/U9KF4iywVQT4t4omrlYuMeEcU4iZBqX8/jROfbtH6hLEmK9SKNg1uIU6f4OzijvE8DPMpXKqJg0TbOW37cl9OIsDijjR+co7m9eO0wz0yHwhrH5yF5DcQb2Qj5XyqZuZDxLrUmjYK4B/hb4JM5o7wHge1kKZVRH0qjoj/0HItftbRHlUsjneO3AQNllDbLkGx+eHXneg4rf8yL17RuujFpZoQSxXKQ2JCoOppwyBc1EMBfHi1zNqtVsuWkBHi9WGFrQDpTtphaR1ar6YRF5ivBWrW+rsYxGDQiLDr5y1SaW37elIW7ofE6qjldRwuM/ski6FOCfPjyrbEVci4zpdiRuinSF+3xOPQQxyiNqlBJV66TeBlEvxH7u0RNrMtLwwufh4BRi8YIZNQ+463B95/5coDTxNBedMiVhi5FJ7BTJ9RitVdV310+k6mn3KVJctG5W04ZyGZfvYEw+R9++Yk3jUYIjme6e3kijdaUEizF19/TG5gZdMm8qX17YtLm/daGiSF5VHRCRfSJymKpa/hHlZT9nlSkdFa179erNNS+jUKk9Yl9xsKyyCIV8jg+e3JWYsdxfHGDZmi1DzuvSc2fy9/c+WbMaL8GARO83KycDvZ40IiM/LWm8SK8BT4nIg8Beb6GqfjYzqZqUcrKfs8yUjorIrWVWc7l5SdUkIY4f7bhoF87pYu7RExNHJH39xdKxevv6WXzX5prG7oQFtqVx/zeCRmXkpyWNgrnffVSEO83aAPSq6jkiMh24E5gIPAF8RFX3i8gY4PvAycAfgEWq+qK7j2uBvwEGgM+q6tpK5amGcrKfs8qUhugcpiTCGr2H1XXJ56Qsg7CXwFe2PCJcdMqUIdMLz8Xe3dPLVas3pXJD11K5xAW2NWNJhCz/Z7UgMVVAVW8F7gB6cBTCHe6ytFwBPO17/zXgBlU9DtiNozhwn3er6luAG9ztcCvoXQjMBN4L/G9XadWdctqClNtCpBwq7fF86JhRrPzQrFJqQmchj4QkBA0MJBef9ugQR5604ewvrji79Hju+rMibRcL53TRWafUhfGjczVL1ag3Wf7PakGakplnAd8FnsNxDkwXkb9V1Z+m+OxRwNnAV4DPiYgAZwB/5W5yK7AM+DZwvvsa4G7gf7nbnw/cqaqvAy+IyDbg7cAvUn7HmlFO9nMWmdIe3gWQtiiRx57+4pC78PwV60IVySDpbS9jR3VE2iiClNsutl6er9eKg2WVx2gmm0eW/7NakCbZ8RvA6ap6mqq+EzgdZ4SRhhuBz3NwFP5GoE9VvVDSHYD3y3QBL4FTNQ+nqNUb/ctDPlNCRC4XkQ0ismHnzp0pxSuPcrKfs8iU9rNwThf/9OFZZY1kgn+6uLucV+c1iX3FQbp7ekuJm3HxIEvPnZla1u6e3sgkwVpHnJSjpMttz5s1Wf/PqiWNDeZVVd3me/888GrSh0TkHPezG0XkNG9xyKaasC7uMwcXqN4E3ASOmzpJvkoox9BXqVGwnLtjOSOZsD9dnC3Hi5ZN4wK+evVmrlq1iSM7C1x0ypRhniAvJibuuwe/9669r0e6t99yxHi2vbq3pu5vT0kmnf9ms3k0q/HZI01Fu28DRwOrcS7sDwFbgUcBVPXeiM9dD3wEOACMBd4A/BBYALzZre17KrBMVReIyFr39S/cwuK/xamet8Q9zvXufkvbRcncqnEw5Rbj9khqi+Fd4EF7R1RluHyHsPJDs4bEmixbsyWVXcZzN3vFoNL84ctNYciikFSUmzx4/rOoQtgORMXBpJkijQV+B7wTOA3YieMBOpeYKF9VvVZVj1LVaThG2nWqejHwMHCBu9mlwI/c12vc97jr17klItYAF4rIGNcDdRzwyxRytxzL79tSVvN6j6T5tgIPPzN82rhwThcrL5g1xDbSWcgPUS7edpuWnpnK6NpfHOCOx14q625abs/rSpVLV2eB+cdODF3nyZ10/uPq7RjDSVMy82M1PuY1wJ0i8mUcz5TXX+lm4AeuEXcXjlJCVbeIyGrg1zijoU+ramtl66Wgu6c3cioStJUEh/HBhvVh9Pb1M3/FutALftzoUaFZvcHjnDNrcqrWHd50zR+TAdHD+Ho0cRMoRedGjUKippn+81+L9rwjibr0plbVR4BH3NfP43iBgtu8hjP9Cvv8V3A8UW1L3CjFf3cMC6y6Z2PvkGZdUQSDsOKCtIDY47zc109HCk9Tf3GAq1ZtGnJBB+WQiJq6cYRNk+KimP3nMMr2FOU583+22W0ezYaVa2gS4uwoNy6aPcS1HHVxDKqm6iLobRulILrcCyrsOP48nWrLP3j7mhbRuiMKAd5x7ETWP7+bAdVSwN7t67enOodRtq40NhgjnGoarxkZ093TGzsaWLZmS8lLEzVC8T6bJnbE2zbNlCCI//jBu3maEU3UvspBgSe27xnyPe7Z2MthEekKnYX8MJuSX27/KCSYqjBmVBozpRFFXD2Yz8V9UFW/UXtxRh7e3TTuwvTn3dSjFceRMSMYgDlffGCIzabSEY24n5lQRu9mcEZgYcbYsfmO0B7Ny84bHn8TF/b/mi9psq+/WJfcnmYK3qslcer5UPcxF6dcZpf7+ARwQvaijQzq5UFJi2ewXLxgRmRA2263BEMwyCzYJSGpCJPifP+l584kn0sXPlfI5yKV8e59RcaM6mDCuHzFof9JfaWyoNmC92pJpIJR1eWquhw4HDhJVa9W1atxkhGPqpeA7U5WOSOdhXzJlpJETmTYBblwTlcqZRa8+BbO6eLRJWfwwoqzU0Uav9zXX3KXJ8nryRe3XV9/kdeKg9ywaHZFfYQakdvTCKVWL9LYYKYC+33v9wPTMpFmBFJpZnQSM488lBf/4Eyp4mwjcUbMrpSyRV18fltH5H7kYBTtwjldsQZffxGouKlYNZG1jcjtafaExWpIo2B+APxSRH6IM6p9P05ZBaNKunt62bd/eIX/WthZHn1uV+l1lHLpLORZdt7MyAsxLOYjjKAbPRijE4cqXH3XwVKYUXVlOgv5Ifs+rJBnbL4jdexQWhoR59LsCYvVkKZcw1eAj+GUVugDPqaqX81asHbHm3cHL5DOQp4bFs3mxRVnR0bOThiXJ1+Bc8M/Fbpx0Ww2LT0z9i7v2VTiInj9F1+YLeG29dsTR0EDg8ry+7YAsOy8meQDJSTyHcI5syYP2bc3FYqSrdKLM4tum0k0e8JiNaR1U48D/qiqt4jIJBGZrqovZClYuxNl3B0/ZlQpCG5vyOgGYM++YkWtUb0ex+Xy+oHwo3UFvB3lGqz9eIo2yoUcZaeI8hxVc3HWu7BUOwfvpakHsxTHkzQDuAXIA7cB87MVrXVJ43JMmnevXLs1slp+pZVnK7mrxymNva8PVYC1shmEXeBXRRQz79tX5IZFs1v+4mzGanm1IM0I5v3AHJxqdqjqyyJyaKZStTBpa6RGzbuV6Gjdaqnkrh6nNIIxItUYrJMSKePsFO16cbYDaWby+92sZgUQkfHZitTapHU5xpW9rEa5RP2g84+dWNFF2JlQhc7/3RYvmDHMfpKGfIeEBsP5aWc7RTuTRsGsFpHvAp0ichnwc6w3dSRpXY5+Y2ItGYRhBuD5x07k9stOLXtf3T29/Pm16D7WHt53Wzini0PGxg+K8x3CJfOmDjGiBstDhNEI46tRPWnKNXxdRN4D/BHHDvMPqvpg5pK1KOW4HL2hfVLBqHIJtgd6YvueUqxJFGF2o5Vrt6au2O/tPy7kP2gULhebCrUeaSrafU1Vr0la1kw0Mpu6kqp0Wdlc/HiZy2GKBMKbipXjEUqK3enyKa1WNsYa4URlU6dRME+o6kmBZU+q6ttqLGPNaHS5hnIT17p7emvePCyIADcsmh2qSMaM6qi4aVraY188b6qVQmhjylYwIvJJ4FPAsYC/6PehwH+65S+bkkYrmDiilM+cLz6QaZuOuBov9SAq7SDYB9poTSqpB/NvwE+B63ELb7v8SVV3hX/EiCPKhX3Xhu2ZKhfP2xIVS5I1OZG2zrcxoonLpt7jtm79JrBLVX+jqr8BiiJySr0EbCeiXNj+vKFa4/e2RAXaTRiXT91fqbOQL7uB2oBq5LEPq1P3RqMxpHFTfxv4s+/9XndZS9Pd08v8FeuYvuR+5q9YV5faG7W6WwuUap50FvKRdVs6C/khJQuiYkmWnjszlcvcK97U8w9ncuOi2aXtkyJfPANvWIzM3v0HuK77qbr/FkZ9SKNgRH2GGlUdpMVLbTaqwE+tsmMVSjVPNi09M9J7EzTcLpzTxQdP7ioVgsqJ8MGTu0ru30eXnBGrLLygOs8l/eiSM3hxxdncEKNsvOlZVIxMcUC53U2IbLdiS0Y6BfO8iHxWRPLu4wqc7o4tS6OqloWVZqiUSuTt7unlno29w2rZ+i/mJCUYpgDClE1YMFxfhJ0pqCDbpdiSkW4k8gngW8B1OP+Fh4DLsxQqa+ptcLyu+6nYiveV4skbVdN2/Ogc81esK3msdu99nf5AFF6wONPiBTMSXeb9xYEh7WJPP35Sqk6O5eQqmfG3PUhTD+ZVVb1QVY9Q1Tep6l+pamJv6mamnt35unt6M1EucFDesJq2uQ5h/4HBIVOPfcEQXxf/xZwm3B+c0U+w5kvSFCcu/yrquxmtTaSCEZHPu8//LCLfCj7qJ2LtqWfi3Mq1WzNRLn55/TVtvanJoWNGpQ7cC17MUVOZtERNcdLmX1kSY/sQd6t62n1uzoi1KqhngZ8shvphOT3BPJ3pZTQzC17MtagTHFend+GcrtgGchbd2z5EKhhVvc99vrV+4tSPrBLngpG6aTotlkOHOBevN0KI+g5plcT40blh+4izw6Rt85o0xYmqfWvKpb2ImyLdJyJroh5JOxaRKSLysIg8LSJbXO8TIjJRRB4UkWfd5wnucnGnX9tE5EkROcm3r0vd7Z8VkUtr8cWzIMz9XesI3UEllTs3rb0jnxv+F4izwxw2NjkoL80Ux8ovjAzicpHe6b78APBmnDKZABcBL6rq38fuWGQyMFlVn3Ar4G0EFgIfxYkMXiEiS4AJqnqNiJwFfAY4CzgF+KaqniIiE3GmaXNxrq2NwMmqujvq2I3KRaplVnRcI3c/cbk8/tFUmkzn4BQr7DNe0mSwc0AaL5LRvpSdi6Sq/9f94JdU9S99q+4TkX9POqCqvgK84r7+k4g8jdMZ8nzgNHezW4FHgGvc5d93g/rWi0inq6ROAx708p9E5EHgvcAdSTLUg7QXcTnkcxJZjzdInI3HPw2MU369ff0s9rUOAStRadSGNIF2k0TkGO+NiEwH4pvdBBCRaTh1fR8D3uQqH08JHeFu1gW85PvYDg62qw1bHjzG5SKyQUQ27Ny5sxzxKiY4JaoZSup8H8/WkZT6kDRlKg4qV67aVPqslag0akEaBXMV8IiIPCIijwAPA1emPYCIHALcA1ypqn+M2zRkmcYsH7pA9SZVnauqcydNKkv/VUw1bTriKA4qqiT2a/Yu+DDbz5WrNjF7+QOhfaPj8Bcpb4SNpBE5YkZ2pCmZ+TMROQ443l30jKq+nmbnIpLHUS63q+q97uLfichkVX3FnQJ5QXs7gCm+jx8FvOwuPy2w/JE0x8+aLGur9PUXuXHRbJbft6VkKC7kOxibz9G3rzjE1jF/xbpQRRes+p+mPSscjGOppLdzNaTtyGC0Dmn6Io0DPgccraqXichxIjJDVX+c8DkBbgaeVtVv+FatAS4FVrjPP/It/zsRuRPHyLvHVUJrga963ibgTODa9F8xGyq5s47qEA6kDH4TSe9Kj7PDhPVpjkotSLvPrIjLETMF05qkmSLdgtPw3itLvwP4corPzQc+ApwhIpvcx1k4iuU9IvIs8B73PcBPcJIotwH/B6eaHq5x90vA4+7ji40ueOXdacslrXIBJ9bkuu50x0iKOQkqi7DUgnL3mQVWlKr9SJPseKyqLhKRiwBUtd8dncSiqv9BdKmQd4Vsr8CnI/b1r8C/ppC1LqSxveREIpvOp+X29duZe3RyP6PTj5/Ebeu3R64PKgt/JHNvX/+wgt2NMua2cxP4kUqqxmsiUuBg47VjgVQ2mHYl7o5ayOe4ZN7UqpULOCc8TdmC+598JXJdPiehyiJtiYV6Yp6r9iPNCGYp8DNgiojcjjP1+WiWQjU7UXdar4DTPRtr5/mIU2bdPb1DjMChpNBzzRLX0s5N4EcqsQrGnQo9gxPNOw9nynOFqv6+DrI1LXF5NLV2XUdND8L6L4VRHNSWMpI2i7IzakOsglFVFZFuVT0ZSAHMg4EAAAmcSURBVJ+e2+ZE3Wkhveu6kM8xNt8RO/qImx6Uo8jMSGo0ijQ2mPUi8heZS9JCxHVHjKKzkB9m54iru5JkCylHaZiR1GgUaWwwpwOfEJEXcToKCM7gpmk7O2ZJsPylFww2Nt8RO6Lo6y8yfswoblg0u6Q0lq3ZEtpR0esGEEfacgxmJDUaSRoF877MpWgRospf9hcHUk1XgpGpUc7+5CCAcDsQREf7GkYjiFQwIjIWp+D3W4CngJtVtXZl8VuQWpS/9EemRk2R0tSQMY+L0QrEjWBuBYrA/8MZxZwAXFEPoZqVOLtHZyGfuoG8t5+oaY5AqfdQHOZxMZqdOCPvCap6iap+F7gA+B91kqlpiTKWCrDsvJmMy6exmR/cz+IFMyJTxa0vkNEOxF0RpdvxSJ8aeYRFmgpw8bypLJzTxVc/8LbUJRbAGYFETbnMtWy0A3FTpFki4tVvEaDgvve8SG/IXLomI8nuEbY+qZxkl+XfGG1MZE3eVqZRNXkrISwi16rrG61G2TV5jfpg3iCjnTEF0wSYN8hoV0zB1JGwFANTLEY7YwqmTli9WWMkYgqmTtSi3qyNgIxWwxRMnai23qyNgIxWJF3oqVE1UXEtaeNd4kZAhtGsmIKpE9XWm7WK+0YrYgqmTvg7K1ZSXLvaEZBhNAKzwdSRauJdouoAWzEpo5kxBdMiWMSv0YqYgmkhLOLXaDXMBmMYRma0jIIRkfeKyFYR2SYiSxotj2EYybSEghGRHPAvHCzdeZGInNBYqQzDSKIlFAzwdmCbqj6vqvuBO4Hzq9mhqtKOtXAMo5loFQXTBbzke7/DXVYxAwMDtEpRqii6e3qZv2Id05fcz/wV6+juqV1PbMOoBa3iRYqqjX1wA5HLgcsBpk6dmrjDUaNGccIJrTvLstwkoxVolRHMDmCK7/1RwMv+DVT1JlWdq6pzJ02alGqn48ePr52EdcZyk4xWoFUUzOPAcSIyXURGAxcCaxosU0Ox3CSjFWgJBeO2Tfk7YC3wNLBaVbc0VqrGYrlJRivQEgoGQFV/oqr/TVWPVdWvNFqeRlNtdrZh1INWMfIaASw3yWgFTMG0MJabZDQ7pmASsDq4hlE5pmBisFgTw6iOljHyNgKLNTGM6jAFE4PFmhhGdZiCicFiTQyjOkzBxGCxJoZRHWbkjcFiTQyjOkzBJBBUMp6B15SMYSRjCiYBc1UbRuWYDSYBc1UbRuWYgknAXNWGUTmmYBIwV7VhVI4pmATMVW0YlWNG3gTMVW0YlWMKJgVWFsEwKsOmSIZhZIYpGMMwMsMUjGEYmSHt2D5VRHYCv0mx6eHA7zMWJwtM7vpicidztKoOa0jWlgomLSKyQVXnNlqOcjG564vJXTk2RTIMIzNMwRiGkRkjXcHc1GgBKsTkri8md4WMaBuMYRjZMtJHMIZhZIgpGMMwMmPEKhgRea+IbBWRbSKypAnkmSIiD4vI0yKyRUSucJdPFJEHReRZ93mCu1xE5Fuu/E+KyEm+fV3qbv+siFxaB9lzItIjIj92308Xkcfc468SkdHu8jHu+23u+mm+fVzrLt8qIgvqIHOniNwtIs+45/zUFjnXV7n/j1+JyB0iMrapz7eqjrgHkAOeA44BRgObgRMaLNNk4CT39aHAfwEnAP8ILHGXLwG+5r4+C/gpIMA84DF3+UTgefd5gvt6Qsayfw74N+DH7vvVwIXu6+8An3Rffwr4jvv6QmCV+/oE9zcYA0x3f5tcxjLfCnzcfT0a6Gz2cw10AS8ABd95/mgzn++GXVCNfACnAmt9768Frm20XAEZfwS8B9gKTHaXTQa2uq+/C1zk236ru/4i4Lu+5UO2y0DOo4CHgDOAH7sX4e+BUcFzDawFTnVfj3K3k+D592+XkcxvcC9UCSxv9nPdBbzkKrRR7vle0Mzne6ROkbwfymOHu6wpcIeyc4DHgDep6isA7vMR7mZR36He3+1G4PPAoPv+jUCfqh4IOX5JNnf9Hnf7est8DLATuMWd2n1PRMbT5OdaVXuBrwPbgVdwzt9Gmvh8j1QFIyHLmsJfLyKHAPcAV6rqH+M2DVmmMctrjoicA7yqqhtTyBW3rt6/xyjgJODbqjoH2IszJYqiKeR2bULn40xrjgTGA++LkaHhco9UBbMDmOJ7fxTwcoNkKSEieRzlcruq3usu/p2ITHbXTwZedZdHfYd6frf5wHki8iJwJ8406UagU0S8Ymb+45dkc9cfBuyqs8yeHDtU9TH3/d04CqeZzzXAu4EXVHWnqhaBe4F30MTne6QqmMeB41zr+2gcA9iaRgokIgLcDDytqt/wrVoDeN6JS3FsM97yv3Y9HPOAPe6wfi1wpohMcO94Z7rLao6qXquqR6nqNJxzuE5VLwYeBi6IkNn7Lhe426u7/ELX6zEdOA74ZRYyu3L/FnhJRLzCyu8Cfk0Tn2uX7cA8ERnn/l88uZv3fGdlkGr2B45n4L9wLOhfaAJ5/jvOMPVJYJP7OAtnzvwQ8Kz7PNHdXoB/ceV/Cpjr29f/BLa5j4/VSf7TOOhFOsb9w24D7gLGuMvHuu+3ueuP8X3+C+532Qq8rw7yzgY2uOe7G8cL1PTnGlgOPAP8CvgBjieoac+3pQoYhpEZI3WKZBhGHTAFYxhGZpiCMQwjM0zBGIaRGaZgDMPIDOvsaKRCRDwXLsCbgQGccHuAt6vq/gbItBa4QFX/VO9jG+kwN7VRNiKyDPizqn49sFxw/lODoR+s3fHrchyjemyKZFSFiLzFrU3yHeAJYIqI9PnWXygi33Nfv0lE7hWRDSLySzcqNri/j4vID0VkrVur5LqI40wWkR0i0umu/5hbq2WziNyS9nhGttgUyagFJ+BEsX7ClxMTxreAf1TV9W7G+I+Bt4Zs93Z3+X7gcXEKWf3ZfxwAZyADIjILuAZ4h6ruEpGJZR7PyAhTMEYteE5VH0+x3buBGZ5iACaISEFV+wPbrVXV3QAi0o2TRvGzmOOcgVNMaReA91zG8YyMMAVj1IK9vteDDC0HMNb3WkhnEA4aBr33e4Mb+vYbZkxMezwjI8wGY9QU1/C6W0SOE5EO4P2+1T8HPu29EZHZEbs5U5yaueNw6p88mnDYn+NkB0909zvRtzzN8YyMMAVjZME1OFOah3Bqj3h8GpjvGmN/DVwW8fn/wKnx2wPcoaqb4g6mqk/i1NP9dxHZBKws83hGRpib2mgqROTjwFtV9cpGy2JUj41gDMPIDBvBGIaRGTaCMQwjM0zBGIaRGaZgDMPIDFMwhmFkhikYwzAy4/8DFbblE/+lXL8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 288x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#还可以观察预测值与真值的散点图\n",
    "plt.figure(figsize=(4, 3))\n",
    "plt.scatter(y_train, y_train_pred_lr)\n",
    "plt.plot([-3, 3], [-3, 3], '--k')   #数据已经标准化，3倍标准差即可\n",
    "plt.axis('tight')\n",
    "plt.xlabel('True price')\n",
    "plt.ylabel('Predicted price')\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 岭回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of RidgeCV on test is 1320.2570600537388\n",
      "The RMSE of RidgeCV on train is 1244.4454443666912\n"
     ]
    }
   ],
   "source": [
    "#岭回归／L2正则\n",
    "#class sklearn.linear_model.RidgeCV(alphas=(0.1, 1.0, 10.0), fit_intercept=True, \n",
    "#                                  normalize=False, scoring=None, cv=None, gcv_mode=None, \n",
    "#                                  store_cv_values=False)\n",
    "from sklearn.linear_model import  RidgeCV\n",
    "\n",
    "#1. 设置超参数（正则参数）范围\n",
    "alphas = [ 0.01, 0.1, 1, 10,100]\n",
    "#n_alphas = 20\n",
    "#alphas = np.logspace(-5,2,n_alphas)\n",
    "\n",
    "#2. 生成一个RidgeCV实例，内置交叉验证的岭回归，默认情况下，它执行通用的交叉验证，这是一种有效的留一交叉验证的形式。\n",
    "ridge = RidgeCV(alphas=alphas, store_cv_values=True, scoring=\"neg_mean_squared_error\")  \n",
    "#3. 模型训练\n",
    "ridge.fit(X2_train, y_train)    \n",
    "\n",
    "#4. 预测\n",
    "y_test_pred_ridge = ridge.predict(X2_test)\n",
    "y_train_pred_ridge = ridge.predict(X2_train)\n",
    "\n",
    "\n",
    "# 评估，使用r2_score评价模型在测试集和训练集上的性能\n",
    "#print ('The r2 score of RidgeCV on test is', r2_score(y_test, y_test_pred_ridge))\n",
    "#print ('The r2 score of RidgeCV on train is', r2_score(y_train, y_train_pred_ridge))\n",
    "\n",
    "#评估,使用RMSE评价模型在测试集和训练集上的性能\n",
    "print ('The RMSE of RidgeCV on test is', np.sqrt(mean_squared_error(y_test, y_test_pred_ridge))) \n",
    "print ('The RMSE of RidgeCV on train is', np.sqrt(mean_squared_error(y_train, y_train_pred_ridge))) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXxU5d338c8ve0ggARK2sIQkoCLgFlFki1atW93ootW61IpIe7d37aJ92aftXe/77mPbp7tSd7vautRqFduqCAEENKCsyr5vCVtIgCQkuZ4/5qBjTMIEcubMZL7v12teTuZcM+eXY5jvXNc5c13mnENERBJXUtAFiIhIsBQEIiIJTkEgIpLgFAQiIglOQSAikuBSgi6go/Ly8lxhYWHQZYiIxJVFixbtds7lt7Yt7oKgsLCQioqKoMsQEYkrZraprW0aGhIRSXAKAhGRBKcgEBFJcAoCEZEEpyAQEUlwCgIRkQSnIBARSXBx9z2C47W2soYXl+xgQE4GA3IzvVsG3dIS5hCIiLQqYd4F399Zw69nrqHl8gs9u6V+EAwFXjj0z/nw5/zu6SQnWTBFi4hEQcIEwRWjB/DJU/uxs7qO7fsPs736MNv3e/f3H2bznkPMX7eH2vrGjzwvJcno5/UiwoOiIKxX0T0jNaDfSkTkxCVMEACkJicxqFc3BvXq1mabA3VH2L7/MDv217HNC4nQrY63Nuxl54E6mpo/2q3onpHykWBoGRR9e2SQmqzTMSISmxIqCCLRIyOVHv1SOblfj1a3NzU7KmvqPtKb2L7/MNu8nxdv3sf+Q0c+8pwkg749PnpuYkDOh/cLcjPJyUzFTENQIhJ9CoIOSk4y+udk0j8nk7OG9Gy1zaGGxjaDYunW/fxreR0NTc0feU63tOSwcxUfnqc4GhT9cjJIT0mOxq8oIglGQeCDbmkplPTJpqRPdqvbm5sdew42hIVE2PmK6sOs3H6A3bX1H3tefvf0D4JiQE4m/Y/e9wKkd1aaehUi0mEKggAkJRn53dPJ757OaYNyW21Td6TpgxPbLYNi1c4aZr5fSd2Rj/Yq0lOSWh16+mBIKieTzDT1KkTkoxQEMSojNZnCvCwK87Ja3e6cY/+hIx89oV0dOsG9Y/9h5qzZza6auo9dLtsrK63NoCjIzSQ/O50kXS4rklAUBHHKzOiZlUbPrDRGFuS02uZIU3Obl8tu2nOIN1u5XDY12btcNieTIb27MWViESV9ukfjVxKRgCgIurCOXC579IT2jrDLZV9ZtpO/v7Od/7ighKllxboEVqSLUhAkuPYul91dW88PXlzB/3t1NS8v28GPPz2a0QNbP6chIvHLt494Zva4mVWa2fI2tl9lZkvN7F0zqzCz8X7VIscnLzud33z+TB65qZR9hxq4+oF5/O+M9zjc0BR0aSLSifzs6z8JXNLO9teB05xzpwNfBB71sRY5AReN6Murd03iujGDebh8PZf8spw31+0OuiwR6SS+BYFzrhzY2872Wuc+uKYlC3BttZXg9chI5X+vGcVTt5+LAZ9/ZCH3PLeU6sNHjvlcEYltgZ79M7NrzOx94GVCvYK22k3xho8qqqqqolegfMzY4t788z8ncsekIp6u2MJFP5vNv1fsDLosETkBgQaBc+5559zJwNXAfe20e9g5V+qcK83Pz49egdKqjNRkvnPpKbzw5fH0zk5nyh8W8eU/L6aq5uPfhhaR2BcT1wN6w0jFZpYXdC0SuVEDc3jxK+P41idP4tWVu7jwZ7N5btFWXMtvsYlITAssCMysxLyJcczsTCAN2BNUPXJ8UpOT+PL5Jcz46gSG9cnmG88s4abH32LL3kNBlyYiEfLz8tGngPnASWa21cxuM7OpZjbVazIZWG5m7wIPAJ9z+igZt0r6ZPP0HWO576pTWbxpH5/8RTlPzNvwsbUbRCT2WLy995aWlrqKioqgy5B2bNt/mHufX8asVVWcMTiXH08ezbC+mqZCJEhmtsg5V9ratpg4RyBdS0FuJk/ccja/+NzpbNx9kMt/NZdfvraGhsbmYz9ZRKJOQSC+MDOuPqOAV++axCdH9uPnr63mU7+ey7tb9gddmoi0oCAQX+Vlp/Pr68/g0ZtKqT58hGsfnMd/v7SSQw2Nx36yiESFgkCi4sIRffn3XRO5fsxgHp27gU/+opx5azVNhUgsUBBI1PTISOV/rhnFX6acS0pSEjc8upC7n9U0FSJBUxBI1J1b1JtXvjaBqZOKeXbxVi762Wz+uVzTVIgERUEggchITeaeS0/mhS+PIy87nal/XMS0Py2isqYu6NJEEo6CQAI1siCHF74yjm9fchKvvVfJRT8r55mKLZqmQiSKFAQSuNTkJKaVlfDK1yYwvG8233p2qaapEIkiBYHEjOL8bP46ZSz3XT2SxZv2cfHPy3lsrqapEPGbgkBiSlKS8YVzh/DqXZMYW9yb+15ayeTpb7J6V03QpYl0WQoCiUkDcjN57OZSfnnd6Wzee4jLfzWHX7y2WtNUiPhAQSAxy8y46vQCXv36RC4b1Z9fvLaGK349h3c27wu6NJEuRUEgMa93djq/vO4MHr+llJq6Rq6d/iY//IemqRDpLAoCiRsXnNyXf399IjeeM4TH523g4p+XM3eNpqkQOVEKAokr3TNSue/qkTx9x1jSkpO48bGFfOuZJVQf0jQVIsdLQSBxaczQXsz42gSmlRXzt3e2ceHPZ/PKsh1BlyURcs6xZe8hXRocI7RCmcS95duqufu5pazYfoBLTu3HD686lT49MoIuS1qoqqln3trdzFmzmzlrqqisqeebFw/nKxcMC7q0hNDeCmUKAukSGpuaeWTOBn7x2mrSU5L47uUj+EzpQMws6NISVt2RJhZt2kf5mirmrN7Nyh0HAMjtlsr4kjy27DvMpj0HmXf3BWSlpwRcbdenIJCEsb6qlnv+toy3NuxlXElvfnTNaAb37hZ0WQnBOceaylrKV1cxZ81uFm7YQ92RZlKTjbOG9GTCsHwmDMvj1AE5JCcZizbtY/L0N/nu5afwpQlFQZff5SkIJKE0NzueenszP5rxPo3NzXzz4pO4ddxQkpPUO+hsu2tDwz3lq3czd20Vuw7UA1Ccn8WEYflMHJ7HOUN7t/mJ/7qH57Nx9yFmf7uM9JTkaJaecNoLAvXHpMtJSjJuOGcIF5zch+8+v5z/fvk9/rF0B/dPHsXJ/XoEXV5cq29sYtHGfZR74/wrtn843DOuJI+Jw/IYPyyfgtzMiF5vWlkJNz3+Fs8v3sZ1Ywb7Wbq0Qz0C6dKcc/xj6Q7+68UVVB8+wrTzS/jy+cX69BmhtoZ7UpJCwz0Th+czviSPkQU5x9Xjcs5x5W/mUVN3hNe/UaZem4/UI5CEZWZcedoAxpfkcd9LK/nV62uYsWwH908ezVlDegZdXkzaU1vP3LCre44O9xTlZ3Hd2YOZMCyPc4p6k90JJ3jNjGllxdz5p8XMWLaDT5024IRfUzpOPQJJKG+squTevy1jx4E6bh5byLc+eVLCX7FydLhnztrQG//ybSc23NNRzc2Oi34+m7SUZGZ8dbyu9PKJThaLhKmtb+TH/3yf38/fREFuJj+6dhQTh+cHXVbUOOdYW1n7wTj/wvV7OXykiZQk48whPZk4LI8Jw/KPe7jneDxTsYVvPbuUJ245m/NP7hOVfSYaBYFIK97euJe7n1vK+qqDTD5zIP/nilPI7ZYWdFm+2HuwITTc44317zwQWhu6KC+LCd4b/7nFnTPcczyONDVT9pNZDMjN4Jmp5wVSQ1encwQirTi7sBczvjqB38xcy29nr2P26kr+68qRXDaqX9wPT9Q3hr7MNWfNbuau2c3y7dU4BzmZoS9zTRiWx/hheQzsGRvfsUhNTuL2CUP5wT9W8taGvYwZ2ivokhKKbz0CM3scuAKodM6NbGX7DcDd3o+1wJ3OuSXHel31CMQPK7cf4O7nlrJsWzUXj+jLfVePpG8cTVPhnGNdVS3lq0PDPQvCh3sG9wx96h+ez6goDvd01OGGJsbfP5NRA3N48tYxQZfT5QQyNGRmEwm9wf++jSA4D3jPObfPzC4FfuCcO+dYr6sgEL80NjXz2NwN/OzV1aSlJHHvZafwubMHxWzv4Ohwz9w1oeGeHdWxNdxzPB54Yy0/+dcqXvqP8YwsyAm6nC4lsHMEZlYIvNRaELRo1xNY7pwrONZrKgjEbxt3H+Sevy1lwfq9jC3qzY+uHUVhXlbQZdHQ2OwN94Te+I8O9/TISGG898Y/viSPQb1iY7jneFQfPsL4/zuTiSfl88Dnzwy6nC4lHs4R3Aa8EnQRIgCFeVn8+Uvn8teKLfzvy+9xyS/Lueui4Xxx3FBSkqM3c3touOfgB2/8C9bv4VDDh8M9X79wOBOG5TF6YG7MDvd0VE5mKjeOHcJvZ69jfVUtRfnZQZeUEALvEZjZ+cCDwHjn3J422kwBpgAMHjz4rE2bNnV+sSKt2Fldx3f/vpzX3tvF6IE53D95NKf092+air0HG7ypmj863DM0fLinqBfdM1J9qyFoVTX1jL9/JlefXsD9nx4ddDldRswODZnZaOB54FLn3OpIXlNDQxJtzjleXraD778QmqbizrJivnJBSadMU9HQ2MzizR8O9yzb9uFwz7iSvA9m7Izn4Z7j8b0XlvPUW5sp//b59M/x54tsiSYmh4bMbDDwN+ALkYaASBDMjCtGD2BccR73vbySX89c+8E0FaWFHbvMMXy4Z+6a3cz3hnuSk4wzB+d2yeGe43H7hCL+tHAzj5Rv4HufGhF0OV2en1cNPQWUAXnALuD7QCqAc+63ZvYoMBk4Os7T2FZahVOPQII2a1Ul9z6/nO3VhyOapmLfwQbmrdvNHO/Szu3ecE9h724ffOIfW9y7Sw/3HI+7nn6XV5btZN49F9Arq2t+0S+a9M1ikU5WW9/IT/+1it/N38iAnEz+55qRlJ0Umhrh6HDPXG8Kh6XecE/3jBTvy1yJOdzTUWt21XDRz8v56gUl3HXxSUGXE/cUBCI+WbRpL99+dinrqg5y2ah+1B9p/shwzxmDckNv/MPzGF2QE9WrjrqCO/5Qwfx1e5h3zwXqMZ2gmDxHINIVnDWkFzO+NoEHZq5l+ux1FORmMvnMgYz3hnt66M3rhEwrK+FfK3bx54WbuWNScdDldFnqEYh0ksamZn3i98GNjy5k1a4a5nz7fDJStaDQ8WqvR6C/WpFOohDwx7SyYqpq6nl20dagS+my9JcrIjFtbHFvTh+Uy0Pl62hsag66nC5JQSAiMe3ocpZb9h7mpaU7gi6nS1IQiEjMu/CUvgzvm830Wetobo6v85rxQEEgIjEvKcm4s6yYVbtqeP39yqDL6XIUBCISFz41egADe2bywBtriberHWOdgkBE4kJKchJ3TCrm3S37mb++1YmK5TgpCEQkbnzmrIHkZaczfda6oEvpUhQEIhI3MlKT+dKEocxZs5ulW/cHXU6XoSAQkbhywzmD6ZGRwoNvqFfQWRQEIhJXumekcvN5hfxzxU7WVtYEXU6XoCAQkbhz67ihZKYmM33W+qBL6RIUBCISd3plpXHdmEG88O42tu47FHQ5cU9BICJx6fYJRZjBI+XqFZwoBYGIxKUBuZlcc0YBf3l7C1U19UGXE9cUBCISt+6YVExDUzNPzNsQdClxTUEgInGrOD+by0b25w/zN3Gg7kjQ5cQtBYGIxLU7y4qpqW/kD/M3BV1K3FIQiEhcG1mQw6Th+Tw+dwOHG5qCLicuKQhEJO5NKytmz8EGnq7YEnQpcUlBICJxb8zQXpQO6cnD5es5ouUsO0xBICJxz8yYdn4x2/Yf5oV3twddTtxREIhIl3D+SX04uV93ps9aq+UsO0hBICJdQqhXUMK6qoP8e+XOoMuJKwoCEekyLh/Vn8Le3Xhw1jotZ9kBCgIR6TKSk4w7JhWzdGs1c9fuDrqcuOFbEJjZ42ZWaWbL29h+spnNN7N6M/umX3WISGK59swC+vZI18I1HeBnj+BJ4JJ2tu8Fvgr81McaRCTBpKckc/uEIuav38PizfuCLicu+BYEzrlyQm/2bW2vdM69DWiCEBHpVNePGUxut1T1CiIUF+cIzGyKmVWYWUVVVVXQ5YhIjMtKT+GW8wp57b1drNqp5SyPJS6CwDn3sHOu1DlXmp+fH3Q5IhIHbjmvkG5pyUyftTboUmJexEFgZuPN7Fbvfr6ZDfWvLBGRE5PbLY0bzhnMi0u2s3mPlrNsT0RBYGbfB+4GvuM9lAr80a+iREQ6w5cmFJGSlMRD5TpX0J5IewTXAFcCBwGcc9uB7u09wcyeAuYDJ5nZVjO7zcymmtlUb3s/M9sK3AV812vT43h/ERGRlvr2yGDyWQN5pmIrlQfqgi4nZqVE2K7BOefMzAGYWdaxnuCcu/4Y23cCAyPcv4jIcZk6qYi/vr2Zx+Zu4DuXnRJ0OTEp0h7B02b2EJBrZrcDrwGP+FeWiEjnGNI7iytGD+CPCzZRfUhXq7cmoiBwzv0UeBZ4DjgJ+J5z7td+FiYi0lnuLCvmYEMTv5u/MehSYlKkJ4uzgJnOuW8R6glkmlmqr5WJiHSSU/r34BMn9+GJeRs41NAYdDkxJ9KhoXIg3cwKCA0L3UpoCgkRkbgw7fxi9h06wlNvaTnLliINAnPOHQKuBX7tnLsGGOFfWSIineusIb04Z2gvHilfT0OjlrMMF3EQmNlY4AbgZe+xSK84EhGJCdPOL2HngTqef2dr0KXElEiD4GvAPcDfnHMrvG8Vz/SvLBGRzjdxWB4jC3rw29nradJylh+INAgOAc3A9Wa2FHgRON+3qkREfGBmTCsrYcPug7yyfEfQ5cSMSId3/gR8E1hOKBBEROLSJ0/tR1F+Fg++sY7LR/XHzIIuKXCR9giqnHP/cM5tcM5tOnrztTIRER8kJxlTJxWzcscBZq3WtPYQeRB838weNbPrzezaozdfKxMR8cnVpxcwICeD6Vq4Bog8CG4FTie09OSnvNsVfhUlIuKntJQkbp9YxFsb9/L2xjYXUkwYkQbBad7CMDc75271bl/0tTIRER9dd/ZgemWl8eAbWrgm0iBYYGb6ApmIdBmZacl8cVwhb6yqYsX26qDLCVSkQTAeeNfMVpnZUjNb5l1GKiISt74wtpDs9BSmz0rscwWRXj56ia9ViIgEICczlRvPHcLD5evYsPsgQ/OOudRKlxTpNNSbWrv5XZyIiN9uGz+U1OQkHpqduL2CiBevFxHpivK7p/PZ0kE8t3grO6oPB11OIBQEIpLwpkwsotnBo3M2BF1KIBQEIpLwBvXqxlWnDeDPCzez72BD0OVEnYJARASYWlbM4SNNPPHmxqBLiToFgYgIMLxvdy4e0ZffvbmR2vrEWs5SQSAi4pl2fgnVh4/w54WJdVGkgkBExHP6oFzGlfTm0TkbqDvSFHQ5UaMgEBEJM62shMqaep5bnDjLWSoIRETCnFfcm9MG5fLQ7PU0NiXGOlwKAhGRMKHlLIvZvPcQLy9LjOUsFQQiIi1cdEpfhvXJ5sE31tGcAIvcKwhERFpISjLuLCtm1a4aZr5fGXQ5vvMtCMzscTOrNLPlbWw3M/uVma31prY+069aREQ66lOnDWBgz0wemLUW57p2r8DPHsGTtD999aXAMO82BZjuYy0iIh2SmpzEHROLeGfzfhas79rLWfoWBM65cqC9o3cV8HsXsgDINbP+ftUjItJRnykdRF52Og/O6trLWQZ5jqAA2BL281bvsY8xsylmVmFmFVVVVVEpTkQkIzWZ28YPZc6a3Szb2nWXswwyCKyVx1odiHPOPeycK3XOlebn5/tclojIh248dzDdM1K6dK8gyCDYCgwK+3kgsD2gWkREWtU9I5WbxxbyzxU7WVtZG3Q5vggyCF4EbvKuHjoXqHbOJca3N0Qkrtw6rpD0lCR+20WXs/Tz8tGngPnASWa21cxuM7OpZjbVazIDWA+sBR4BpvlVi4jIieidnc51Zw/m7+9sY9v+rrecZYpfL+ycu/4Y2x3wZb/2LyLSmaZMLOKPCzbxSPl6fnDlqUGX06n0zWIRkQgMyM3kmjMK+Mvbm9ldWx90OZ1KQSAiEqGpZcXUNzbzxLyutci9gkBEJELF+dlcOrIfv5+/iQN1R4Iup9MoCEREOmBaWQk1dY38cUHXWc5SQSAi0gEjC3KYODyfx+d2neUsFQQiIh00rayY3bUNPF2x5diN44CCQESkg84Z2ouzhvTkodnrOdIFlrNUEIiIdNDR5Sy37T/Mi+/G/8w4CgIRkeNwwcl9OLlfd6bPjv/lLBUEIiLHwSy0nOXaylr+vXJX0OWcEAWBiMhxunxUf4b07sb0OF/OUkEgInKcUpKTuGNiMUu2VjNv7Z6gyzluCgIRkRMw+awC+nSP7+UsFQQiIicgPSWZ2ycU8ea6PbyzeV/Q5RwXBYGIyAn6/DmDyclM5cFZ8blwjYJAROQEZaWncMt5hby6cherdtYEXU6HKQhERDrBLecV0i0tOS6Xs1QQiIh0gp5ZaXx+zGBeXLKdLXsPBV1OhygIREQ6yZcmFJFsxkPl8dUrUBCIiHSSfjkZTD6rgKcrtlJZUxd0ORFTEIiIdKI7JhbT2NTMY3PjZzlLBYGISCcqzMvi8tED+NOCzVQfio/lLBUEIiKd7M5JxdTWN/L7+RuDLiUiCgIRkU42YkAPLji5D0+8uZFDDY1Bl3NMCgIRER9MKytm78EG/vJW7C9nqSAQEfFBaWEvxgztxSNz1tPQGNvLWSoIRER8Mq2smB3Vdfz9nW1Bl9IuBYGIiE8mDc/n1AE9+O3sdTTF8HKWvgaBmV1iZqvMbK2Z3dPK9iFm9rqZLTWzWWY20M96RESiKbTIfQnrdx/kn8t3Bl1Om3wLAjNLBh4ALgVGANeb2YgWzX4K/N45Nxr4IfAjv+oREQnCJSP7UZSXxYMxvJylnz2CMcBa59x651wD8BfgqhZtRgCve/ffaGW7iEhcS04ypk4qZsX2A8xeXRV0Oa3yMwgKgPDrprZ6j4VbAkz27l8DdDez3i1fyMymmFmFmVVUVcXmgRQRacvVZxTQPycjZheu8TMIrJXHWvaLvglMMrN3gEnANuBj375wzj3snCt1zpXm5+d3fqUiIj5KS0ni9glFvLVhLxUb9wZdzsf4GQRbgUFhPw8Etoc3cM5td85d65w7A7jXe6zax5pERAJx3ZhB9MpKi8legZ9B8DYwzMyGmlkacB3wYngDM8szs6M1fAd43Md6REQC0y0thVvPK2Tm+5Ws3H4g6HI+wrcgcM41Al8B/gW8BzztnFthZj80syu9ZmXAKjNbDfQF/sevekREgnbT2EKy01OYHmPLWab4+eLOuRnAjBaPfS/s/rPAs37WICISK3K6pXLDuYN5pHw937hoOIV5WUGXBOibxSIiUXXb+KGkJCfF1HKWCgIRkSjq0z2Dz5YO5LlF29hZHRvLWSoIRESi7I6JxTQ5x6Nz1gddCqAgEBGJukG9unHlaQP481ub2XewIehyFAQiIkG4s6yYQw1NPPnmxqBLURCIiARheN/uXDSiL0++uZHa+mCXs1QQiIgEZFpZMdWHj/DUws2B1qEgEBEJyBmDe3JecW8embOe+samwOpQEIiIBGhaWQmVNfU8tyi45SwVBCIiARpX0pvTBubwUPk6GpuCWeReQSAiEiAz486yEjbtOcTLy3YEUoOCQEQkYBeP6EtJn2ymz1oXyHKWCgIRkYAlJRl3Tirm/Z01zHy/Mvr7j/oeRUTkY648fQAFuZk88Eb0F7lXEIiIxIDU5CTumFTE4s37WbghustZKghERGLEZ0sHkZcd/eUsFQQiIjEiIzWZL44fSvnqKpZtjd7y7QoCEZEYcuO5Q+iekcL02Wujtk8FgYhIDOmRkcpNY4fwyvKdrK2sjco+FQQiIjHm1nFDSU9J4qEoLXKvIBARiTF52elcd/Zgnn9nG9v2H/Z9fwoCEZEYdPvEIgAeKfd/OUsFgYhIDCrIzeTqMwr4y9ub2VNb7+u+FAQiIjFq6qRi6hubeWLeRl/3oyAQEYlRJX2yueTUfvxu/kZq6o74th8FgYhIDJtWVkJNXSN/XODfcpYKAhGRGDZqYA4ThuXx2NwN1B3xZzlLBYGISIybVlbC7tp6nqnY4svrKwhERGLcuUW9uPK0AeR2S/Pl9X0NAjO7xMxWmdlaM7unle2DzewNM3vHzJaa2WV+1iMiEo/MjF9dfwafOm2AL6/vWxCYWTLwAHApMAK43sxGtGj2XeBp59wZwHXAg37VIyIirfOzRzAGWOucW++cawD+AlzVoo0Denj3c4DtPtYjIiKt8DMICoDwMxtbvcfC/QC40cy2AjOA/2jthcxsiplVmFlFVVWVH7WKiCQsP4PAWnms5UKc1wNPOucGApcBfzCzj9XknHvYOVfqnCvNz8/3oVQRkcTlZxBsBQaF/TyQjw/93AY8DeCcmw9kAHk+1iQiIi34GQRvA8PMbKiZpRE6GfxiizabgU8AmNkphIJAYz8iIlHkWxA45xqBrwD/At4jdHXQCjP7oZld6TX7BnC7mS0BngJucc61HD4SEREfpfj54s65GYROAoc/9r2w+yuBcX7WICIi7bN4+wBuZlXApuN8eh6wuxPL6SyxWhfEbm2qq2NUV8d0xbqGOOdavdom7oLgRJhZhXOuNOg6WorVuiB2a1NdHaO6OibR6tJcQyIiCU5BICKS4BItCB4OuoA2xGpdELu1qa6OUV0dk1B1JdQ5AhER+bhE6xGIiEgLCgIRkQTXpYPAzH5iZu97i948b2a5bbRrdwEdH+r6jJmtMLNmM2vzUjAz22hmy8zsXTOriKG6onq8vH32MrNXzWyN99+ebbRr8o7Xu2bWckqTzqrlWAsupZvZX73tC82s0I86jqOuW8ysKuz4fClKdT1uZpVmtryN7WZmv/LqXmpmZ8ZIXWVmVh12vL7XWjsf6hrkLdj1nvfv8WuttOncY+ac67I34GIgxbt/P3B/K22SgXVAEZAGLAFG+FzXKcBJwCygtJ12G4G8KB6vY9YVxPHy9vtj4B7v/j2t/b/0ttX6XMcxf39gGvBb7/51wF+jcCD9hI0AAAYcSURBVHwiqesW4DfR+nsK2+9E4ExgeRvbLwNeITRj8bnAwhipqwx4KYDj1R8407vfHVjdyv/LTj1mXbpH4Jz7twvNeQSwgNAMqC1FsoBOZ9f1nnNulZ/7OB4R1hX14+W5Cvidd/93wNVR2GdrIvn9w2t9FviEmbU2LXu06wqEc64c2NtOk6uA37uQBUCumfWPgboC4Zzb4Zxb7N2vITRXW8u1XDr1mHXpIGjhi4QStKVIFtAJigP+bWaLzGxK0MV4gjpefZ1zOyD0DwXo00a7DG8RowVm5kdYRPL7f9DG+yBSDfT2oZaO1gUw2RtKeNbMBrWyPQix/G9wrJktMbNXzOzUaO/cG1Y8A1jYYlOnHjNfJ52LBjN7DejXyqZ7nXMveG3uBRqBP7X2Eq08dsLX1EZSVwTGOee2m1kf4FUze9/7FBNkXb4cL2i/tg68zGDvmBUBM81smXNuXWfU54nk9/ftGLUjkn3+A3jKOVdvZlMJ9Vou8LmuSARxvCKxmND8PLVmdhnwd2BYtHZuZtnAc8B/OucOtNzcylOO+5jFfRA45y5sb7uZ3QxcAXzCeYNrLUSygE6n1xXha2z3/ltpZs8T6v6fUBB0Ql2+HC9ovzYz22Vm/Z1zO7wucGUbr3H0mK03s1mEPk11ZhBE8vsfbbPVzFIIrcft9xDEMetyzu0J+/ERQufNYoFvf1MnIvzN1zk3w8weNLM855zvk9GZWSqhEPiTc+5vrTTp1GPWpYeGzOwS4G7gSufcoTaaRbKATtSZWZaZdT96n9CJ71avboiyoI7Xi8DN3v2bgY/1Xsysp5mle/fzCE1xvrKT64jk9w+v9dPAzDY+hES1rhZjyFcSGnuOBS8CN3lXwpwLVB8dBgySmfU7em7HzMYQer/c0/6zOmW/BjwGvOec+1kbzTr3mEX7jHg0b8BaQuNo73q3o1dyDABmhLW7jNCZ+XWEhkj8rusaQoleD+wC/tWyLkJXfyzxbitipa4gjpe3z97A68Aa77+9vMdLgUe9++cBy7xjtgy4zadaPvb7Az8k9IEDQivtPeP9/b0FFEXpGB2rrh95f0tLgDeAk6NU11PADuCI9/d1GzAVmOptN+ABr+5ltHMlXZTr+krY8VoAnBelusYTGuZZGvbedZmfx0xTTIiIJLguPTQkIiLHpiAQEUlwCgIRkQSnIBARSXAKAhGRBKcgkIRhZrUn+PxnvW8st9dmlrUzc2ukbVq0zzezf0baXqSjFAQiEfDmmUl2zq2P9r6dc1XADjMbF+19S2JQEEjC8b6N+RMzW26h9R4+5z2e5E0jsMLMXjKzGWb2ae9pNxD2bWYzm+5NbrfCzP6rjf3Umtn/M7PFZva6meWHbf6Mmb1lZqvNbILXvtDM5njtF5vZeWHt/+7VINLpFASSiK4FTgdOAy4EfuJNv3AtUAiMAr4EjA17zjhgUdjP9zrnSoHRwCQzG93KfrKAxc65M4HZwPfDtqU458YA/xn2eCVwkdf+c8CvwtpXABM6/quKHFvcTzonchzGE5qFswnYZWazgbO9x59xzjUDO83sjbDn9Aeqwn7+rDc1eIq3bQShKQHCNQN/9e7/EQifPOzo/UWEwgcgFfiNmZ0ONAHDw9pXEprqQ6TTKQgkEbW1SEx7i8ccJjSHEGY2FPgmcLZzbp+ZPXl02zGEz+dS7/23iQ//HX6d0BxPpxHqrdeFtc/wahDpdBoakkRUDnzOzJK9cfuJhCaHm0to4ZYkM+tLaKnCo94DSrz7PYCDQLXX7tI29pNEaPZRgM97r9+eHGCH1yP5AqHlJ48aTmzMPitdkHoEkoieJzT+v4TQp/RvO+d2mtlzwCcIveGuJrQqVLX3nJcJBcNrzrklZvYOoZkp1wPz2tjPQeBUM1vkvc7njlHXg8BzZvYZQrODHgzbdr5Xg0in0+yjImHMLNuFVqTqTaiXMM4LiUxCb87jvHMLkbxWrXMuu5PqKgeucs7t64zXEwmnHoHIR71kZrlAGnCfc24ngHPusJl9n9C6sJujWZA3fPUzhYD4RT0CEZEEp5PFIiIJTkEgIpLgFAQiIglOQSAikuAUBCIiCe7/AwSkjNGr/VnXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "alpha is: 1.0\n"
     ]
    }
   ],
   "source": [
    "mse_mean = np.mean(ridge.cv_values_, axis = 0)\n",
    "plt.plot(np.log10(alphas), mse_mean.reshape(len(alphas),1)) \n",
    "\n",
    "#这是为了标出最佳参数的位置，不是必须\n",
    "#plt.plot(np.log10(ridge.alpha_)*np.ones(3), [0.28, 0.29, 0.30])\n",
    "\n",
    "plt.xlabel('log(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show()\n",
    "\n",
    "print ('alpha is:', ridge.alpha_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef_lr</th>\n",
       "      <th>coef_ridge</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>workingday</td>\n",
       "      <td>1.820107e+17</td>\n",
       "      <td>252.951084</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>1.820107e+17</td>\n",
       "      <td>142.910080</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>holiday</td>\n",
       "      <td>1.820107e+17</td>\n",
       "      <td>-320.635605</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>1.820107e+17</td>\n",
       "      <td>-75.225559</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>atemp</td>\n",
       "      <td>4.750216e+03</td>\n",
       "      <td>4222.787191</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>2.989119e+02</td>\n",
       "      <td>416.739566</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>2.487798e+02</td>\n",
       "      <td>954.982085</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>2.243891e+02</td>\n",
       "      <td>-27.344726</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>9.346169e+01</td>\n",
       "      <td>25.023829</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>9.283011e+01</td>\n",
       "      <td>271.793644</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>8.209855e+01</td>\n",
       "      <td>276.196781</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>1.843760e+01</td>\n",
       "      <td>584.594257</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>-2.095820e+02</td>\n",
       "      <td>-247.300064</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>-2.415052e+02</td>\n",
       "      <td>57.657211</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>-3.376975e+02</td>\n",
       "      <td>232.734920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>-4.075142e+02</td>\n",
       "      <td>373.308622</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>-4.827059e+02</td>\n",
       "      <td>-184.288275</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-5.167331e+02</td>\n",
       "      <td>-220.854537</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>-5.363376e+02</td>\n",
       "      <td>-213.071568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_3</td>\n",
       "      <td>-6.614880e+02</td>\n",
       "      <td>-888.315421</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_2</td>\n",
       "      <td>-7.356910e+02</td>\n",
       "      <td>-451.214156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-7.377327e+02</td>\n",
       "      <td>-225.578227</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-1.270310e+03</td>\n",
       "      <td>-315.312430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-1.383459e+03</td>\n",
       "      <td>-1749.924960</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>season_1</td>\n",
       "      <td>-1.394206e+03</td>\n",
       "      <td>-1351.094022</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-1.413671e+03</td>\n",
       "      <td>-1350.702043</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>hum</td>\n",
       "      <td>-2.884003e+03</td>\n",
       "      <td>-2189.270650</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns       coef_lr   coef_ridge\n",
       "26    workingday  1.820107e+17   252.951084\n",
       "21     weekday_6  1.820107e+17   142.910080\n",
       "25       holiday  1.820107e+17  -320.635605\n",
       "16     weekday_0  1.820107e+17   -75.225559\n",
       "22         atemp  4.750216e+03  4222.787191\n",
       "14  weathersit_1  2.989119e+02   416.739566\n",
       "10        mnth_9  2.487798e+02   954.982085\n",
       "19     weekday_4  2.243891e+02   -27.344726\n",
       "20     weekday_5  9.346169e+01    25.023829\n",
       "6         mnth_5  9.283011e+01   271.793644\n",
       "5         mnth_3  8.209855e+01   276.196781\n",
       "11       mnth_10  1.843760e+01   584.594257\n",
       "17     weekday_1 -2.095820e+02  -247.300064\n",
       "18     weekday_3 -2.415052e+02    57.657211\n",
       "7         mnth_6 -3.376975e+02   232.734920\n",
       "9         mnth_8 -4.075142e+02   373.308622\n",
       "13       mnth_12 -4.827059e+02  -184.288275\n",
       "4         mnth_2 -5.167331e+02  -220.854537\n",
       "12       mnth_11 -5.363376e+02  -213.071568\n",
       "2       season_3 -6.614880e+02  -888.315421\n",
       "1       season_2 -7.356910e+02  -451.214156\n",
       "3         mnth_1 -7.377327e+02  -225.578227\n",
       "8         mnth_7 -1.270310e+03  -315.312430\n",
       "15  weathersit_3 -1.383459e+03 -1749.924960\n",
       "0       season_1 -1.394206e+03 -1351.094022\n",
       "24     windspeed -1.413671e+03 -1350.702043\n",
       "23           hum -2.884003e+03 -2189.270650"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names2), \"coef_lr\":list((lr.coef_.T)), \"coef_ridge\":list((ridge.coef_.T))})\n",
    "fs.sort_values(by=['coef_lr'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Lasso"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of RidgeCV on test is 1306.66288078546\n",
      "The RMSE of RidgeCV on train is 1248.8858263745103\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\hp\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_split.py:1978: FutureWarning: The default value of cv will change from 3 to 5 in version 0.22. Specify it explicitly to silence this warning.\n",
      "  warnings.warn(CV_WARNING, FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "#### Lasso／L1正则\n",
    "# class sklearn.linear_model.LassoCV(eps=0.001, n_alphas=100, alphas=None, fit_intercept=True, \n",
    "#                                    normalize=False, precompute=’auto’, max_iter=1000, \n",
    "#                                    tol=0.0001, copy_X=True, cv=None, verbose=False, n_jobs=1,\n",
    "#                                    positive=False, random_state=None, selection=’cyclic’)\n",
    "from sklearn.linear_model import LassoCV\n",
    "\n",
    "#1. 设置超参数搜索范围\n",
    "#alphas = [ 0.01, 0.1, 1, 10,100]\n",
    "\n",
    "#2. 生成学习器实例\n",
    "#lasso = LassoCV(alphas=alphas)\n",
    "\n",
    "#1. 设置超参数搜索范围\n",
    "#Lasso可以自动确定最大的alpha，所以另一种设置alpha的方式是设置最小的alpha值（eps） 和 超参数的数目（n_alphas），\n",
    "#然后LassoCV对最小值和最大值之间在log域上均匀取值n_alphas个\n",
    "# np.logspace(np.log10(alpha_max * eps), np.log10(alpha_max),num=n_alphas)[::-1]\n",
    "\n",
    "#2 生成LassoCV实例（默认超参数搜索范围）\n",
    "lasso = LassoCV()  \n",
    "\n",
    "#3. 训练（内含CV）\n",
    "lasso.fit(X2_train, y_train)  \n",
    "\n",
    "#4. 测试\n",
    "y_test_pred_lasso = lasso.predict(X2_test)\n",
    "y_train_pred_lasso = lasso.predict(X2_train)\n",
    "\n",
    "\n",
    "# 评估，使用r2_score评价模型在测试集和训练集上的性能\n",
    "#print 'The r2 score of LassoCV on test is', r2_score(y_test, y_test_pred_lasso)\n",
    "#print 'The r2 score of LassoCV on train is', r2_score(y_train, y_train_pred_lasso)\n",
    "#评估,使用RMSE评价模型在测试集和训练集上的性能\n",
    "print ('The RMSE of RidgeCV on test is', np.sqrt(mean_squared_error(y_test, y_test_pred_lasso))) \n",
    "print ('The RMSE of RidgeCV on train is', np.sqrt(mean_squared_error(y_train, y_train_pred_lasso))) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZ4AAAEICAYAAABvQ5JRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deZxddX3/8dd7tiyE7JMQspAACRCWBBhCAEE2IVhLEEGCViKiEauPn7TWilZLRVu1LrSoYLEggaJAESFiaAwBNLKETCABQoBMFkjIvu8zmZnP74/7HbgZZjKT5c6d5f18PO7jnvv5Lud7uCGffM/53nMUEZiZmbWUgnwPwMzMOhYnHjMza1FOPGZm1qKceMzMrEU58ZiZWYty4jEzsxaVs8QjqbOkFyTNkzRf0rdTfKakuem1QtIjKX6upM1ZZf+c1dc4SW9IqpB0Y1Z8mKRZkhZKekBSSYp3Sp8rUvnQrDZfT/E3JF2cq+M3M7OGFeWw70rg/IjYJqkY+IukxyPi7LoKkn4LPJrVZmZEfCS7E0mFwM+BDwHLgdmSpkTEa8APgFsi4n5JvwCuA25P7xsj4mhJE1K9qySNBCYAxwOHA09IGhERNY0dRN++fWPo0KEH+J/CzKxjmTNnzrqIKG2oLGeJJzK/TN2WPhan17u/VpV0KHA+cG0TXY0BKiJicWp3PzBe0oLU/hOp3mTgX8gknvFpG+Ah4GeSlOL3R0QlsERSRer/ucZ2PnToUMrLy5txxGZmVkfSW42V5fQaj6RCSXOBNcD0iJiVVfxRYEZEbMmKnZFOzT0u6fgUGwgsy6qzPMX6AJsiorpefI82qXxzqt9YX2Zm1kJymngioiYiRgODgDGSTsgqvhr4TdbnF4EjImIU8FPgkRRXQ13vJb6/bd4laZKkcknla9eubaCJmZntrxZZ1RYRm4CngXEAkvqQOcX1h6w6WyJiW9qeChRL6ktmVjI4q7tBwApgHdBTUlG9ONltUnkPYMNe+qo/3jsioiwiykpLGzxFaWZm+ymXq9pKJfVM212AC4HXU/GVwGMRsSur/mHpOgySxqSxrQdmA8PTCrYSMosDpqRrSE8BV6QuJvLeQoUp6TOp/MlUfwowIa16GwYMB144+EdvZmaNyeWqtgHA5LQqrQB4MCIeS2UTgO/Xq38F8AVJ1cBOYEJKFtWSvgRMAwqBuyJifmrzNeB+Sd8FXgLuTPE7gXvT4oENaX9ExHxJDwKvAdXAF/e2os3MzA4++bEIe1dWVhZe1WZmtm8kzYmIsobKfOcCMzNrUU48Zmb2Pnf9ZQnT5q/KSd9OPGZmtoddu2v48R/f4MkFa3LSvxOPmZnt4YkFq9leVcP40YfnpH8nHjMz28Ojc1fQv3snTj+yT076d+IxM7N3bd6xm6ffWMNfn3Q4hQUN3ezlwDnxmJnZu6a+upLdNcH40bm7jaUTj5mZvevRue9wZOkhnDCwe8724cRjZmYArNy8k1lLNjB+1EDSHcxywonHzMwA+P28FUSQs9VsdZx4zMwMgCnzVjBqcE+G9j0kp/tx4jEzM9Zs2cWr72xh3PGH5XxfTjxmZsYzi9YBcPbwvjnflxOPmZkxc+E6enUtZuSA3K1mq+PEY2bWwUUEz1Ss48yj+1KQox+NZnPiMTPr4Bat3cbqLZV84Ojcn2aD3D76urOkFyTNkzRf0rdT/G5JSyTNTa/RKS5Jt0qqkPSypFOy+pooaWF6TcyKnyrpldTm1qxHZ/eWND3Vny6pV1P7MDPrqGYuzFzfafOJB6gEzo+IUcBoYJyksansqxExOr3mptglwPD0mgTcDpkkAtwEnA6MAW6qSySpzqSsduNS/EZgRkQMB2akz43uw8ysI3umYh1H9OnK4N5dW2R/OUs8kbEtfSxOr709Z3s8cE9q9zzQU9IA4GJgekRsiIiNwHQySWwA0D0inovM87vvAS7L6mty2p5cL97QPszMOqTdNbU8v3gDZ7XQbAdyfI1HUqGkucAaMsljVir613Sq6xZJnVJsILAsq/nyFNtbfHkDcYD+EbESIL33a2IfZmYd0rxlm9hWWc3Z7SXxRERNRIwGBgFjJJ0AfB04FjgN6A18LVVvaClF7Ed8b5rVRtIkSeWSyteuXdtEl2ZmbdfMheuQ4IyjcvPsnYa0yKq2iNgEPA2Mi4iV6VRXJfArMtdtIDP7GJzVbBCwoon4oAbiAKvrTqGl97rntzbWV/3x3hERZRFRVlpauo9Ha2bWdjxTsY6TBvagZ9eSFttnLle1lUrqmba7ABcCr2clBJG59vJqajIFuCatPBsLbE6nyaYBF0nqlRYVXARMS2VbJY1NfV0DPJrVV93qt4n14g3tw8ysw9lWWc3cZZs4swVPswEU5bDvAcBkSYVkEtyDEfGYpCcllZI57TUXuD7Vnwp8GKgAdgDXAkTEBknfAWanejdHxIa0/QXgbqAL8Hh6AXwfeFDSdcDbwJV724eZWUf0wpL1VNdGiy2jrpOzxBMRLwMnNxA/v5H6AXyxkbK7gLsaiJcDJzQQXw9csC/7MDPraJ6pWE9JUQGnHtGr6coHke9cYGbWQT1TsY6yI3rRubiwRffrxGNm1gGt21bJ66u2tujvd+o48ZiZdUDPLloPwJktuIy6jhOPmVkH9GzFOg7tVMSJA3u0+L6deMzMOqBnFq3j9CP7UFTY8mnAicfMrINZtmEHyzbs5KyjW/40GzjxmJl1OM9UZB6DkI+FBeDEY2bW4TyzaD2lh3ZieL9uedm/E4+ZWQcSETy3aB1nHdWH9OzMFufEY2bWgcxfsYV126rydpoNnHjMzDqUJxasRoLzju3XdOUcceIxM+tAZixYw8mDe9K3W6emK+eIE4+ZWQexessuXnlnMxcc1z+v43DiMTPrIGYsyDwT80InHjMzawkzFqxmUK8ujOifn2XUdZx4zMw6gJ1VNfylYh0XHtc/b8uo6zjxmJl1AM9UrKOyupYLjsvfarY6OUs8kjpLekHSPEnzJX07xe+T9IakVyXdJak4xc+VtFnS3PT656y+xqU2FZJuzIoPkzRL0kJJD0gqSfFO6XNFKh+a1ebrKf6GpItzdfxmZq3JjNdX061TEacPy8/92bLlcsZTCZwfEaOA0cA4SWOB+4BjgROBLsBns9rMjIjR6XUzgKRC4OfAJcBI4GpJI1P9HwC3RMRwYCNwXYpfB2yMiKOBW1I9UrsJwPHAOOC21L+ZWbtVWxvMWLCGc0b0paQo/ye6cjaCyNiWPhanV0TE1FQWwAvAoCa6GgNURMTiiKgC7gfGK3OS8nzgoVRvMnBZ2h6fPpPKL0j1xwP3R0RlRCwBKlL/Zmbt1tzlm1iztZILjs3varY6OU19kgolzQXWANMjYlZWWTHwKeD/spqckU7NPS7p+BQbCCzLqrM8xfoAmyKiul58jzapfHOq31hfZmbt1u/nraCkqIAPHd8BEk9E1ETEaDKzmjGSTsgqvg34c0TMTJ9fBI5Ip+Z+CjyS4g0tv4i9xPe3zbskTZJULql87dq1DTQxM2sbamqDx15eyXnHlNK9c3G+hwO00Kq2iNgEPE3mugqSbgJKgb/PqrOl7tRcREwFiiX1JTMrGZzV3SBgBbAO6CmpqF6c7DapvAewYS991R/vHRFRFhFlpaWl+3/gZmZ5NmvJetZureSvRx2e76G8K5er2kol9UzbXYALgdclfRa4GLg6Imqz6h+WrsMgaUwa23pgNjA8rWArIbM4YEq6RvQUcEXqYiLwaNqekj6Typ9M9acAE9Kqt2HAcDLXmczM2qXfz1tB15LCVnN9B6Co6Sr7bQAwOa0aKwAejIjHJFUDbwHPpTzzcFrBdgXwhVS+E5iQkkW1pC8B04BC4K6ImJ/28TXgfknfBV4C7kzxO4F7JVWQmelMAIiI+ZIeBF4DqoEvRkRNDv8bmJnlTVV1LY+/uooPjexPl5LWs4BXmb/brTFlZWVRXl6e72GYme2zJ19fzWfuLufOiWUtfmNQSXMioqyhsvwv6DYzs5z4/byV9OhSzNnDW9e1aiceM7N2aGdVDX+cv4pLTjisVfxoNFvrGo2ZmR0Uf3xtFdurarh0dOtZzVbHicfMrB369ay3OaJPV8a2gnuz1efEY2bWzixau41ZSzZw1WmDKSjI7yMQGuLEY2bWzjwwexlFBeKKU5u6FWZ+OPGYmbUjldU1PDRnOR8a2Z9+h3bO93Aa5MRjZtaO/HH+ajZsr2LCmCH5HkqjnHjMzNqR37zwNgN7duHso/vmeyiNcuIxM2snlqzbzrOL1nP1mNa5qKCOE4+ZWTvxq2eWUFwoPl42uOnKeeTEY2bWDmzYXsWD5cu4bPRA+nVvnYsK6jjxmJm1A/c+9xa7dtcy6Zwj8z2UJjnxmJm1cbt213DPc0s5/9h+DO9/aL6H0yQnHjOzNu6hOctZv72qTcx2wInHzKxNq6kN/nvmYkYN6sHpw3rnezjNkstHX3eW9IKkeZLmS/p2ig+TNEvSQkkPpMdZkx5H/YCkilQ+NKuvr6f4G5IuzoqPS7EKSTdmxfd5H2ZmbdG0+atYun4Hk845ivRU51YvlzOeSuD8iBgFjAbGSRoL/AC4JSKGAxuB61L964CNEXE0cEuqh6SRZB5dfTwwDrhNUmF6pPbPgUuAkcDVqS77ug8zs7aopjb4yfQ3Oar0EMadcFi+h9NsOUs8kbEtfSxOrwDOBx5K8cnAZWl7fPpMKr9AmfQ9Hrg/IiojYglQAYxJr4qIWBwRVcD9wPjUZl/3YWbW5vzupXeoWLONf7joGApb8Q9G68vpNZ40M5kLrAGmA4uATRFRnaosBwam7YHAMoBUvhnokx2v16axeJ/92IeZWZtSWV3DLdPf5MSBPdrUbAdynHgioiYiRgODyMxQjmuoWnpvKF3HQYzvbR97kDRJUrmk8rVr1zbQxMwsvx6YvYx3Nu3kqxcf02au7dRpkVVtEbEJeBoYC/SUVJSKBgEr0vZyYDBAKu8BbMiO12vTWHzdfuyj/njviIiyiCgrLS3dv4M2M8uRHVXV3DqjgtOH9ebs4a33ZqCNyeWqtlJJPdN2F+BCYAHwFHBFqjYReDRtT0mfSeVPRkSk+IS0Im0YMBx4AZgNDE8r2ErILECYktrs6z7MzNqMu59dyrptlW1ytgNQ1HSV/TYAmJxWnxUAD0bEY5JeA+6X9F3gJeDOVP9O4F5JFWRmIRMAImK+pAeB14Bq4IsRUQMg6UvANKAQuCsi5qe+vrYv+zAzays27aji9qcXceFx/Sgb2jZ+t1Of/A/+vSsrK4vy8vJ8D8PMDIDvTV3AHTMX8/iXz+bYw7rneziNkjQnIsoaKvOdC8zM2oiVm3dy97NL+ejJA1t10mmKE4+ZWRvxH9MXEgF/d+GIfA/lgDjxmJm1ARVrtvK/c5bxybFDGNy7a76Hc0CceMzM2oD/eGIhXYoL+dJ5R+d7KAfMicfMrJWrWLONP7yykmvOHEqfbp3yPZwD5sRjZtbK/fypCjoXFfLZDwzL91AOCiceM7NWbOm67Tw69x0+efqQdjHbASceM7NW7banKyguLGgzTxdtDiceM7NWatmGHTz84jtcPWYI/bp3zvdwDhonHjOzVuq2pxdRIPH5D7af2Q448ZiZtUpvrd/O/5YvY8KYwQzo0SXfwzmonHjMzFqh/3hiIUWFahe/26nPicfMrJV5Y9VWHpn7DhPPHNquru3UceIxM2tlfjL9DbqVFHH9OUfleyg54cRjZtaKvLx8E9Pmr+a6s4fR65CSfA8nJ5x4zMxakR9Oe4NeXYu5rp3cpaAhuXz09WBJT0laIGm+pC+n+AOS5qbXUklzU3yopJ1ZZb/I6utUSa9IqpB0q9KzXiX1ljRd0sL03ivFlepVSHpZ0ilZfU1M9RdKmoiZWSvx7KJ1zFy4jr8992gO7Vyc7+HkTLMTj6QPSLo2bZdKaiodVwNfiYjjgLHAFyWNjIirImJ0RIwGfgs8nNVmUV1ZRFyfFb8dmAQMT69xKX4jMCMihgMz0meAS7LqTkrtkdQbuAk4HRgD3FSXrMzM8iki+Pf/e4MBPTrzqTOOyPdwcqpZiUfSTcDXgK+nUDHwP3trExErI+LFtL0VWAAMzOpTwMeB3zSx7wFA94h4LjLP6b4HuCwVjwcmp+3J9eL3RMbzQM/Uz8XA9IjYEBEbgem8l8TMzPLmj6+tZu6yTdxw4XA6Fxfmezg51dwZz0eBS4HtABGxAji0uTuRNBQ4GZiVFT4bWB0RC7NiwyS9JOlPks5OsYHA8qw6y3kvgfWPiJVpTCuBflltljXQprG4mVne1NQGP5z2BkeWHsLHThmU7+HkXFEz61VFREgKAEmHNHcHkrqROaV2Q0RsySq6mj1nOyuBIRGxXtKpwCOSjgfUQLfR1G4badOsviRNInOKjiFDhjSxKzOzA/Pwi8upWLON2z95CkWF7X/NV3OP8EFJ/0XmlNXngCeAXzbVSFIxmaRzX0Q8nBUvAi4HHqiLRURlRKxP23OARcAIMrOS7H8CDAJWpO3V6RRa3Sm5NSm+HBjcQJvG4nuIiDsioiwiykpLS5s6TDOz/batspof/fENRg3qwbgTDsv3cFpEsxJPRPwIeIhMEjkG+OeI+One2qRrOHcCCyLiJ/WKLwRej4jlWfVLJRWm7SPJLAxYnE6hbZU0NvV5DfBoajYFqFuZNrFe/Jq0um0ssDn1Mw24SFKvtKjgohQzM8uL/5j+Jmu2VvIvlx5PWrDb7jXrVFs6tfZkREyXdAxwjKTiiNi9l2ZnAZ8CXqlbMg18IyKmAhN4/6KCc4CbJVUDNcD1EbEhlX0BuBvoAjyeXgDfJzMbuw54G7gyxacCHwYqgB3AtQARsUHSd4DZqd7NWfswM2tRC1Zu4VfPLmXCaUM4eUjHWWCrzEKxJipJc8gsBugFPA+UAzsi4pO5HV7+lZWVRXl5eb6HYWbtTG1tcOV/PceSdduZ8fcfbHd3KZA0JyLKGipr7jUeRcQOMtdlfhoRHwVGHqwBmpl1NA+9uJw5b23kxkuObXdJpynNTjySzgA+CfwhxZq7Is7MzLKs2bqLf5u6gLIjenFFB1g+XV9zE8+XydwV4OGImJ/uWvBk7oZlZtY+RQTf/N2r7Kiq4fsfO4mCgo6xoCBbc2ctO4Ba4GpJf0Pm9zBNXxwyM7M9TJm3gj++tpqvX3IsR/frlu/h5EVzE899wD8Ar5JJQGZmto/WbN3FTVPmc/KQnnz27CPzPZy8aW7iWRsRv8/pSMzM2rlvPZI5xfbDK0ZR2AFPsdVpbuK5SdJ/k7kDdGVdMPtuBGZm1rgnX1/NtPmr+cdxx3TYU2x1mpt4rgWOJXNX6rpTbcGejzQwM7MG7Npdw79MeY2jSg/hsx/ouKfY6jQ38YyKiBNzOhIzs3bqv/60mLc37OC+z55OSVH7vwloU5r7X+B5Sf7BqJnZPlq2YQe3PV3BX500gLOO7pvv4bQKzZ3xfACYKGkJmWs8AiIiTsrZyMzM2oFv//41CgvEN//quHwPpdVobuLxUzrNzPbRsxXreGJBZkHBgB5d8j2cVqNZiSci3sr1QMzM2pPa2uBfpy5gYM8ufOasYfkeTqviq1xmZjnw6Lx3mL9iC1+9+Bg6FxfmezitihOPmdlBtmt3DT+a9iYnDOzOpaMOz/dwWh0nHjOzg+xXzyzlnU07+caHj+uQNwFtihOPmdlBtGXXbm57uoILju3HmUd5+XRDcpZ4JA2W9JSkBZLmS/pyiv+LpHckzU2vD2e1+bqkCklvSLo4Kz4uxSok3ZgVHyZplqSFkh6QVJLindLnilQ+tKl9mJkdDL+Z9TZbd1Xzdx8ake+htFq5nPFUA1+JiOOAscAXs36EektEjE6vqQCpbAJwPJnl27dJKpRUCPwcuITMU0+vzurnB6mv4cBG4LoUvw7YGBFHA7ekeo3uI3f/CcysI6mqruVXzyzlrKP7cMLAHvkeTquVs8QTESsj4sW0vRVYAAzcS5PxwP0RURkRS4AKYEx6VUTE4oioAu4HxksScD7wUGo/Gbgsq6/Jafsh4IJUv7F9mJkdsMdeXsGqLbv4XAd+5EFztMg1nnSq62RgVgp9SdLLku6S1CvFBgLLspotT7HG4n2ATRFRXS++R1+pfHOq31hfZmYHJCK448+LOab/oXxwRGm+h9Oq5TzxSOoG/Ba4ISK2ALcDRwGjgZXAj+uqNtA89iO+P33VH/MkSeWSyteuXdtAEzOzPc1cuI7XV23lc+ccSeYEizUmp4lHUjGZpHNf3bN7ImJ1RNRERC3wS9471bUcGJzVfBCwYi/xdUBPSUX14nv0lcp7ABv20tceIuKOiCiLiLLSUv/Lxcya9suZi+nfvZN/t9MMuVzVJuBOYEFE/CQrPiCr2kfJPE4bYAowIa1IGwYMB14AZgPD0wq2EjKLA6ZERABPAVek9hOBR7P6mpi2rwCeTPUb24eZ2X57bcUWZi5cx8Qzh/qxB83Q3JuE7o+zgE8Br0iam2LfILMqbTSZU1xLgc8DRMR8SQ8Cr5FZEffFiKgBkPQlYBpQCNwVEfNTf18D7pf0XeAlMomO9H6vpAoyM50JTe3DzGx//XLmYg4pKeSTpx+R76G0CcpMBKwxZWVlUV5enu9hmFkrtWLTTs7596eYeOZQvvURP7asjqQ5EVHWUJnnhGZmB+BXzywhgGvPGprvobQZTjxmZvtpy67d/OaFZXzkpAEM6tU138NpM5x4zMz2069nvc22ymr/YHQfOfGYme2HzO1xlnDmUb49zr5y4jEz2w/3z36b1Vsq+fwHj8r3UNocJx4zs320vbKaW2csZMyw3pwz3I8+2FdOPGZm++i/Zy5h3bYqbrzkWN8eZz848ZiZ7YP12yq548+LuPj4/pwypFfTDex9nHjMzPbBT5+sYOfuGr568bH5Hkqb5cRjZtZMb63fzn2z3uLjZYM5ul+3fA+nzXLiMTNrhojgW4/Op6SwgBsu9GOtD4QTj5lZMzz28kr+/OZa/uHiYzisR+d8D6dNc+IxM2vC5p27ufmx1zhxYA+uOWNovofT5uXysQhmZu3CD6e9zvptldw18TQKC7x8+kB5xmNmthflSzdw36y3+fSZwzhxkG+NczA48ZiZNWLLrt3c8MBcBvXqwt9f5AUFB0suH309WNJTkhZImi/pyyn+Q0mvS3pZ0u8k9UzxoZJ2SpqbXr/I6utUSa9IqpB0a3qsNpJ6S5ouaWF675XiSvUq0n5OyeprYqq/UNJEzMwaEBF883evsnLzLv5zwsl06+QrEwdLLmc81cBXIuI4YCzwRUkjgenACRFxEvAm8PWsNosiYnR6XZ8Vvx2YBAxPr3EpfiMwIyKGAzPSZ4BLsupOSu2R1Bu4CTgdGAPcVJeszMyy/e6ld5gybwU3XDDcdyg4yHKWeCJiZUS8mLa3AguAgRHxx4ioTtWeBwbtrR9JA4DuEfFcZJ7TfQ9wWSoeD0xO25Prxe+JjOeBnqmfi4HpEbEhIjaSSYJ1SczMDICl67bzrUdeZczQ3vzteUfnezjtTotc45E0FDgZmFWv6DPA41mfh0l6SdKfJJ2dYgOB5Vl1lqcYQP+IWAmZRAf0y2qzrIE2jcXNzADYums3n7unnKLCAn5y1SivYsuBnJ+0lNQN+C1wQ0RsyYr/E5nTcfel0EpgSESsl3Qq8Iik44GGvvVoareNtGlWX5ImkTlFx5AhQ5rYlZm1FzW1wQ33z2Xxuu3c85kxfpx1juR0xiOpmEzSuS8iHs6KTwQ+AnwynT4jIiojYn3angMsAkaQmZVkn44bBKxI26vTKbS6U3JrUnw5MLiBNo3F9xARd0REWUSUlZaW7s+hm1kb9MNpbzDj9TXc9NcjOetoP2cnV3K5qk3AncCCiPhJVnwc8DXg0ojYkRUvlVSYto8kszBgcTqFtlXS2NTnNcCjqdkUoG5l2sR68WvS6raxwObUzzTgIkm90qKCi1LMzDq4B8uX8Ys/LeITpw/hU2OPyPdw2rVcnmo7C/gU8IqkuSn2DeBWoBMwPa2Kfj6tYDsHuFlSNVADXB8RG1K7LwB3A13IXBOquy70feBBSdcBbwNXpvhU4MNABbADuBYgIjZI+g4wO9W7OWsfZtZBTX1lJTf+9mXOHt6Xb196vB/ulmNKZ7qsEWVlZVFeXp7vYZhZjjz1xhom3VPOSYN6cu91Y+ha4t/rHAyS5kREWUNlvnOBmXVYzy5ax/X3zmFE/0O569OnOem0ECceM+uQps1fxad/NZshvbtyz2fG0KNLcb6H1GE48ZhZh/PA7Lf5wv/MYeSA7jz4+TPo061TvofUoXheaWYdRm1t8NMnK7jliTc5Z0Qpv/ibU3x6LQ/8X9zMOoRtldV85cG5TJu/mstPHsj3P3YSJUU+6ZMPTjxm1u4tWruNz987hyXrtvOtj4zkM2cN9ZLpPHLiMbN2q7Y2uPvZpfz7tNfpWlLEvdeN4cyjfEeCfHPiMbN26a312/nqQy/zwpINnHdMKd+7/CQO69E538MynHjMrJ3ZUVXN7U8v4r/+vJhORQX88IqTuOLUQT611oo48ZhZuxARPPbySv5t6gJWbt7FZaMP58ZLjvMspxVy4jGzNu/Vdzbz7d/PZ/bSjYwc0J1brz6Z04b2zvewrBFOPGbWZq3esouf/PFNHpyzjN5dS/je5Sfy8bLBfnhbK+fEY2ZtzrbKau740yJ+OXMJ1bW1XHfWMP7fhcPp3tm3vWkLnHjMrM2ICH730jt87/HXWbu1ko+cNIB/vPhYhvTxk0LbEiceM2sTXluxhZumvMrspRsZNbgnd3zqVE4e0ivfw7L94MRjZq3aO5t2csv0N3n4xeX07FrCDz52IleeOpgCX8dps3L56OvBkp6StEDSfElfTvHekqZLWpjee6W4JN0qqULSy5JOyeprYqq/UNLErPipkl5JbW5Nj8ber32YWeuys6qGf5u6gPN+9DRT5q7gM2cN48mvfJCrThvipNPG5fIOedXAVyLiOGAs8EVJI4EbgRkRMRyYkT4DXAIMT69JwO2QSSLATcDpwBjgprpEkupMymo3LsX3aR9m1rosXL2V8T//C7+cuZhLRx3OU189l/cNuNoAABCYSURBVG9+ZCQ9u5bke2h2EOQs8UTEyoh4MW1vBRYAA4HxwORUbTJwWdoeD9wTGc8DPSUNAC4GpkfEhojYCEwHxqWy7hHxXGSe331Pvb72ZR9m1gpEBA+WL+Ovf/YXNmyv4p7PjOFHV45iYM8u+R6aHUQtco1H0lDgZGAW0D8iVkImOUnql6oNBJZlNVueYnuLL28gzn7sY+UBHJ6ZHQRrtu7iW4+8yrT5qznjyD7854TR9Ovuuw60RzlPPJK6Ab8FboiILXu5X1JDBbEf8b0OpzltJE0icyqOIUOGNNGlmR2IiGDKvBXcNGU+O6pquPGSY/nc2Uf6R6DtWE4Tj6RiMknnvoh4OIVXSxqQZiIDgDUpvhwYnNV8ELAixc+tF386xQc1UH9/9rGHiLgDuAOgrKysqWRmZvvp5eWb+Nc/LGDWkg2cPKQnP7xiFEf365bvYVmO5XJVm4A7gQUR8ZOsoilA3cq0icCjWfFr0sqzscDmdLpsGnCRpF5pUcFFwLRUtlXS2LSva+r1tS/7MLMW9Pb6HXz5/pe49GfPULFmG98ZfzwPXX+mk04HkcsZz1nAp4BXJM1NsW8A3wcelHQd8DZwZSqbCnwYqAB2ANcCRMQGSd8BZqd6N0fEhrT9BeBuoAvweHqxr/sws5axbMMOfvZkBQ+9uJyiAvHF847i+g8exaG+1U2HosyCMGtMWVlZlJeX53sYZm3aqs27uPXJhTw4exkFBeITY4bwt+ce5cUD7ZikORFR1lCZ71xgZjmzZusu7vjTYu55/i0iggljBvOl84b7GTkdnBOPmR10r6/awp0zl/Do3BVU19Zy+SmD+PIFwxnc2zfzNCceMztIqqprmTZ/Fb+e9TbPLV5P5+ICrjptMJ/5wDCG9T0k38OzVsSJx8wOyKrNu7j3+aU8MHsZ67ZVMahXF/5x3DF8YswQ3+LGGuTEY2b75eXlm/jvmUuY+spKaiK48Lj+fPL0IZwzvNQ38bS9cuIxs2aLCJ6pWM/tf6rgmYr1HNqpiE+fOZSJZw719RtrNiceM2vSrt01TJm3gsnPLmX+ii30O7QT3/jwsVw9Zoh/g2P7zInHzBq1aUcVd/5lCf/z/Fts3LGb4f268b3LT+TyUwbSqagw38OzNsqJx8zeZ+uu3fzqmaX88s+L2VZVzYeO68+nzxzKGUf1YS83+jVrFiceM3tXVXUtv571Frc+WcGG7VVcNLI/f3/RCI49rHu+h2btiBOPmRERPP7qKn7wf6/z1vodnHlUH7427lhGDe6Z76FZO+TEY9bBvbl6Kzc9Op/nFq/nmP6H8qtrT+PcEaU+pWY548Rj1kFt3F7Fz56qYPKzSzmkUxHfvewErh4zxA9gs5xz4jHrYDbv3M2dMxdz51+WsGN3DRNOG8xXLz6W3of4LgPWMpx4zDqIFZt2MvnZpfz6hbfZuquaD594GDdcOIIR/Q/N99Csg3HiMWvHamqD5xat5/7Zb/P4q6uICC45YQB/e95RHH94j3wPzzooJx6zdiYiePWdLUx9dSWPvPQOKzfvonvnIq77wDCuOeMIBvXyrW0sv3KWeCTdBXwEWBMRJ6TYA8AxqUpPYFNEjJY0FFgAvJHKno+I61ObU3nv8dZTgS9HREjqDTwADAWWAh+PiI3KLMX5TzKPuN4BfDoiXkx9TQS+mfbx3YiYnJODN2thm3ZU8fziDfylYi1PvLaGVVt2UVggPjiilH/6q+O48Lj+dC72nQasdcjljOdu4GfAPXWBiLiqblvSj4HNWfUXRcToBvq5HZgEPE8m8YwDHgduBGZExPcl3Zg+fw24BBieXqen9qenRHUTUAYEMEfSlIjYeFCO1qyFVNfUsmjtduYt38TLyzfx4lubWLBqCxHQtaSQc4aXcuHI/px3TCl9unXK93DN3idniSci/pxmMu+TZiUfB87fWx+SBgDdI+K59Pke4DIyiWc8cG6qOhl4mkziGQ/cExEBPC+pZ+rnXGB6RGxIfU0nk8R+s7/HuDcLVm7hs5PLKS4UxYUFmVdRASVZn0uKCijJeu9UXECnogI6FRXSuTjrvbiQTkUFdCkupHNxIV1KCt/d7lxcQOdU3rm4kOLCglwcjrWwiGD99iqWbdjBW+t3sHT9dhav3c6bq7eyeO12qmpqAejWqYiTBvXg7y4cwRlH9WHUoJ6UFPnPgLVu+brGczawOiIWZsWGSXoJ2AJ8MyJmAgOB5Vl1lqcYQP+IWAkQESsl9UvxgcCyBto0Fn8fSZPIzLIYMmTIvh8dcEhJEWOP7MPumlqqa2upqq5ld02wu6aW3TW1bK+qoaq6lqrqGnbXBJXVNVRW11K5u5Zd1TVE7NduKSxQSkp1yayQksICCgtEcaEoLBBFhQUUFSjFMmV1nzPvBRQW8O57UUHDdYoK1Wi8U3ZSrbfdqagwJdjMdl28o/x+JCLYXlXDmi27WLVlF6u37GLFpl0s27CD5Rt38s6mzKuquvbdNhIc3qMLI/p344MjSjnmsEM5aVBPjux7iJ99Y21OvhLP1ew501gJDImI9emaziOSjgca+j+qqb+SG2vT7L4i4g7gDoCysrL9SgFD+nTlxx8ftT9NiYh3k9Gu3bXs2l2zx/bO3Zntnbtr2FW1Z9murO2q6tpMMquuoaY202d1bW3armVHVVBTG1TXBtU1tdRE+lyTea+JFK8NagN219RSm8aWC0UFmYRVN8PLTlidigrenel1KSmiS3EBXUuK3o11LcnMALvWzQZLCulaXJjqFKQ2mfJORQUH9Kv8iHj3HwnbqqrZXlnNtspqNu/czZb02tzgq5qN26vYsKNqj6RSp2+3Egb16srIw7tz0cj+HN6zCwN7dmFo364M6tXV12is3WjxxCOpCLgcOLUuFhGVQGXaniNpETCCzKxkUFbzQcCKtL1a0oA02xkArEnx5cDgBtos571Tc3Xxpw/OUR1ckigpEiVFBRzaOd+jaVhNbbz72l1bS21KYDW1kZnJ1dSmGd1725XV7yXDunjl7vc+18363r+ded+5u4aN23ezc3cNO6qq2VmVScL7kwhLigronJJbUZqlFRWIAom6nPRe8s0k6rpZa3NnpF2KC+nepYgeXYrp0aWYgT07c8Lh3el9SAm9DymhX/dO9D+0M/26d+bwnp3pWuJFptYx5ONP+oXA6xHx7ik0SaXAhoiokXQkmYUBiyNig6StksYCs4BrgJ+mZlOAicD30/ujWfEvSbqfzOKCzSk5TQP+TVKvVO8i4Os5PdJ2rDCdXgPoQn7/Jb67JpOUdlbVvJuMdlTVsCu976iqzswUq2rYkWaLldU1VO7OJL/qmlqqazKJM4DaCAj2OI1YUvTedbnOWbOybp2KOKRTEd06FdE9JZgeXYrp3qXIz6sxa0Qul1P/hswMo6+k5cBNEXEnMIH3X9A/B7hZUjVQA1xftwgA+ALvLad+PL0gk3AelHQd8DZwZYpPJbOUuoLMcuprAVIS+w4wO9W7OWsf1obVLdbo7idhmrUJiv29it1BlJWVRXl5eb6HYWbWpkiaExFlDZV53aWZmbUoJx4zM2tRTjxmZtainHjMzKxFOfGYmVmLcuIxM7MW5cRjZmYtyr/jaYKktcBb+R7HAeoLrMv3IA4SH0vr1F6Opb0cB+T/WI6IiNKGCpx4OgBJ5Y39kKut8bG0Tu3lWNrLcUDrPhafajMzsxblxGNmZi3KiadjuCPfAziIfCytU3s5lvZyHNCKj8XXeMzMrEV5xmNmZi3KiacdkTRO0huSKiTd2EB5J0kPpPJZkoa2/CibpxnH8mlJayXNTa/P5mOcTZF0l6Q1kl5tpFySbk3H+bKkU1p6jM3VjGM5V9LmrO/kn1t6jM0habCkpyQtkDRf0pcbqNMmvpdmHkvr+14iwq928AIKgUXAkUAJMA8YWa/O3wK/SNsTgAfyPe4DOJZPAz/L91ibcSznAKcArzZS/mEyDzcUMBaYle8xH8CxnAs8lu9xNuM4BgCnpO1DgTcb+PPVJr6XZh5Lq/tePONpP8YAFRGxOCKqgPuB8fXqjAcmp+2HgAskqQXH2FzNOZY2ISL+DOztSbfjgXsi43mgp6QBLTO6fdOMY2kTImJlRLyYtrcCC4CB9aq1ie+lmcfS6jjxtB8DgWVZn5fz/j+A79aJiGpgM9CnRUa3b5pzLAAfS6dBHpI0uGWGdtA191jbijMkzZP0uKTj8z2YpqTTzScDs+oVtbnvZS/HAq3se3HiaT8amrnUX7LYnDqtQXPG+XtgaEScBDzBezO5tqatfCfN8SKZ26SMAn4KPJLn8eyVpG7Ab4EbImJL/eIGmrTa76WJY2l134sTT/uxHMj+V/8gYEVjdSQVAT1onadOmjyWiFgfEZXp4y+BU1tobAdbc763NiEitkTEtrQ9FSiW1DfPw2qQpGIyf1HfFxEPN1ClzXwvTR1La/xenHjaj9nAcEnDJJWQWTwwpV6dKcDEtH0F8GSkq4+tTJPHUu98+6Vkzm23RVOAa9IqqrHA5ohYme9B7Q9Jh9VdM5Q0hszfL+vzO6r3S2O8E1gQET9ppFqb+F6acyyt8XspyufO7eCJiGpJXwKmkVkVdldEzJd0M1AeEVPI/AG9V1IFmZnOhPyNuHHNPJb/J+lSoJrMsXw6bwPeC0m/IbOqqK+k5cBNQDFARPwCmEpmBVUFsAO4Nj8jbVozjuUK4AuSqoGdwIRW+g+bs4BPAa9Impti3wCGQJv7XppzLK3ue/GdC8zMrEX5VJuZmbUoJx4zM2tRTjxmZtainHjMzKxFOfGYmVmLcuIxyxFJ2w6w/UOSjmyiztOSyg60Tr36pZL+r7n1zfaVE49ZK5Tup1UYEYtbet8RsRZYKemslt63dQxOPGY5ln79/kNJr0p6RdJVKV4g6bb0HJXHJE2VdEVq9kng0aw+bpdUnup+u5H9bJP0Y0kvSpohqTSr+EpJL0h6U9LZqf5QSTNT/RclnZlV/5E0BrODzonHLPcuB0YDo4ALgR+mW/5cDgwFTgQ+C5yR1eYsYE7W53+KiDLgJOCDkk5qYD+HAC9GxCnAn8jcWaBOUUSMAW7Iiq8BPpTqXwXcmlW/HDh73w/VrGm+ZY5Z7n0A+E1E1ACrJf0JOC3F/zciaoFVkp7KajMAWJv1+eOSJpH5f3YAMBJ4ud5+aoEH0vb/ANk3jKzbnkMm2UHmdjc/kzQaqAFGZNVfAxy+j8dp1ixOPGa519jD9vb2EL6dQGcAScOAfwBOi4iNku6uK2tC9v2w6u7kXcN7/9//HbCazEysANiVVb9zGoPZQedTbWa592fgKkmF6brLOcALwF/IPMyuQFJ/MjfgrLMAODptdwe2A5tTvUsa2U8BmRtCAnwi9b83PYCVacb1KTI3ZK0zAni1Gcdmts884zHLvd+RuX4zj8ws5B8jYpWk3wIXkPkL/k0yT47cnNr8gUwieiIi5kl6CZgPLAaeaWQ/24HjJc1J/VzVxLhuA34r6UrgqdS+znlpDGYHne9ObZZHkrpFxDZJfcjMgs5KSakLmWRwVro21Jy+tkVEt4M0rj8D4yNi48HozyybZzxm+fWYpJ5ACfCdiFgFEBE7Jd0EDATebskBpdOBP3HSsVzxjMfMzFqUFxeYmVmLcuIxM7MW5cRjZmYtyonHzMxalBOPmZm1KCceMzNrUf8fzZh0MzqYKM0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "alpha is: 5.713237013848561\n"
     ]
    }
   ],
   "source": [
    "mses = np.mean(lasso.mse_path_, axis = 1)\n",
    "plt.plot(np.log10(lasso.alphas_), mses) \n",
    "#plt.plot(np.log10(lasso.alphas_)*np.ones(3), [0.3, 0.4, 1.0])\n",
    "plt.xlabel('log(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show()    \n",
    "            \n",
    "print ('alpha is:', lasso.alpha_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef_lr</th>\n",
       "      <th>coef_ridge</th>\n",
       "      <th>coef_lasso</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>workingday</td>\n",
       "      <td>1.820107e+17</td>\n",
       "      <td>252.951084</td>\n",
       "      <td>149.148471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>1.820107e+17</td>\n",
       "      <td>142.910080</td>\n",
       "      <td>3.070342</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>holiday</td>\n",
       "      <td>1.820107e+17</td>\n",
       "      <td>-320.635605</td>\n",
       "      <td>-256.473157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>1.820107e+17</td>\n",
       "      <td>-75.225559</td>\n",
       "      <td>-122.918926</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>atemp</td>\n",
       "      <td>4.750216e+03</td>\n",
       "      <td>4222.787191</td>\n",
       "      <td>4653.650622</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>2.989119e+02</td>\n",
       "      <td>416.739566</td>\n",
       "      <td>409.416640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>2.487798e+02</td>\n",
       "      <td>954.982085</td>\n",
       "      <td>711.388012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>2.243891e+02</td>\n",
       "      <td>-27.344726</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>9.346169e+01</td>\n",
       "      <td>25.023829</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>9.283011e+01</td>\n",
       "      <td>271.793644</td>\n",
       "      <td>31.110753</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>8.209855e+01</td>\n",
       "      <td>276.196781</td>\n",
       "      <td>259.470171</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>1.843760e+01</td>\n",
       "      <td>584.594257</td>\n",
       "      <td>646.456064</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>-2.095820e+02</td>\n",
       "      <td>-247.300064</td>\n",
       "      <td>-219.804037</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>-2.415052e+02</td>\n",
       "      <td>57.657211</td>\n",
       "      <td>13.251925</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>-3.376975e+02</td>\n",
       "      <td>232.734920</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>-4.075142e+02</td>\n",
       "      <td>373.308622</td>\n",
       "      <td>25.789027</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>-4.827059e+02</td>\n",
       "      <td>-184.288275</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-5.167331e+02</td>\n",
       "      <td>-220.854537</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>-5.363376e+02</td>\n",
       "      <td>-213.071568</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_3</td>\n",
       "      <td>-6.614880e+02</td>\n",
       "      <td>-888.315421</td>\n",
       "      <td>-541.192191</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_2</td>\n",
       "      <td>-7.356910e+02</td>\n",
       "      <td>-451.214156</td>\n",
       "      <td>-181.901856</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-7.377327e+02</td>\n",
       "      <td>-225.578227</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-1.270310e+03</td>\n",
       "      <td>-315.312430</td>\n",
       "      <td>-576.605518</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-1.383459e+03</td>\n",
       "      <td>-1749.924960</td>\n",
       "      <td>-1661.780737</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>season_1</td>\n",
       "      <td>-1.394206e+03</td>\n",
       "      <td>-1351.094022</td>\n",
       "      <td>-1261.191726</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-1.413671e+03</td>\n",
       "      <td>-1350.702043</td>\n",
       "      <td>-1233.257886</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>hum</td>\n",
       "      <td>-2.884003e+03</td>\n",
       "      <td>-2189.270650</td>\n",
       "      <td>-2061.125308</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns       coef_lr   coef_ridge   coef_lasso\n",
       "26    workingday  1.820107e+17   252.951084   149.148471\n",
       "21     weekday_6  1.820107e+17   142.910080     3.070342\n",
       "25       holiday  1.820107e+17  -320.635605  -256.473157\n",
       "16     weekday_0  1.820107e+17   -75.225559  -122.918926\n",
       "22         atemp  4.750216e+03  4222.787191  4653.650622\n",
       "14  weathersit_1  2.989119e+02   416.739566   409.416640\n",
       "10        mnth_9  2.487798e+02   954.982085   711.388012\n",
       "19     weekday_4  2.243891e+02   -27.344726    -0.000000\n",
       "20     weekday_5  9.346169e+01    25.023829     0.000000\n",
       "6         mnth_5  9.283011e+01   271.793644    31.110753\n",
       "5         mnth_3  8.209855e+01   276.196781   259.470171\n",
       "11       mnth_10  1.843760e+01   584.594257   646.456064\n",
       "17     weekday_1 -2.095820e+02  -247.300064  -219.804037\n",
       "18     weekday_3 -2.415052e+02    57.657211    13.251925\n",
       "7         mnth_6 -3.376975e+02   232.734920     0.000000\n",
       "9         mnth_8 -4.075142e+02   373.308622    25.789027\n",
       "13       mnth_12 -4.827059e+02  -184.288275     0.000000\n",
       "4         mnth_2 -5.167331e+02  -220.854537    -0.000000\n",
       "12       mnth_11 -5.363376e+02  -213.071568    -0.000000\n",
       "2       season_3 -6.614880e+02  -888.315421  -541.192191\n",
       "1       season_2 -7.356910e+02  -451.214156  -181.901856\n",
       "3         mnth_1 -7.377327e+02  -225.578227    -0.000000\n",
       "8         mnth_7 -1.270310e+03  -315.312430  -576.605518\n",
       "15  weathersit_3 -1.383459e+03 -1749.924960 -1661.780737\n",
       "0       season_1 -1.394206e+03 -1351.094022 -1261.191726\n",
       "24     windspeed -1.413671e+03 -1350.702043 -1233.257886\n",
       "23           hum -2.884003e+03 -2189.270650 -2061.125308"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names2), \"coef_lr\":list((lr.coef_.T)), \"coef_ridge\":list((ridge.coef_.T)), \"coef_lasso\":list((lasso.coef_.T))})\n",
    "fs.sort_values(by=['coef_lr'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用逐步回归筛选后特征看下线性回归的结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 从原始数据中分离输入特征x和输出y\n",
    "y = df[\"cnt\"]\n",
    "\n",
    "X3 = df[[\"season_1\",\"season_3\", \"mnth_7\", \"mnth_9\", \"mnth_10\", \"weathersit_1\", \"weathersit_3\", \"weekday_6\", \"atemp\", \"hum\", \"windspeed\",\"workingday\"]]\n",
    "#特征名称，用于后续显示权重系数对应的特征，在后面再实现\n",
    "feat_names3 = X3.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(584, 12)"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#将数据分割训练数据与测试数据\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# 随机采样20%的数据构建测试样本，其余作为训练样本\n",
    "X3_train, X3_test, y_train, y_test = train_test_split(X3, y, random_state=33, test_size=0.2)\n",
    "X3_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>atemp</td>\n",
       "      <td>5006.342509</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>859.758899</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>810.153965</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>workingday</td>\n",
       "      <td>373.107640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>329.437499</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>303.592139</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_3</td>\n",
       "      <td>-578.839380</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-658.001081</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>season_1</td>\n",
       "      <td>-1087.531957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-1588.652085</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-1733.883506</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>hum</td>\n",
       "      <td>-2698.945616</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns         coef\n",
       "8          atemp  5006.342509\n",
       "3         mnth_9   859.758899\n",
       "4        mnth_10   810.153965\n",
       "11    workingday   373.107640\n",
       "5   weathersit_1   329.437499\n",
       "7      weekday_6   303.592139\n",
       "1       season_3  -578.839380\n",
       "2         mnth_7  -658.001081\n",
       "0       season_1 -1087.531957\n",
       "10     windspeed -1588.652085\n",
       "6   weathersit_3 -1733.883506\n",
       "9            hum -2698.945616"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 1.使用默认配置初始化学习器实例\n",
    "lr = LinearRegression()\n",
    "\n",
    "# 2.用训练数据训练模型参数\n",
    "lr.fit(X3_train, y_train)\n",
    "\n",
    "# 3. 用训练好的模型对测试集进行预测\n",
    "y_test_pred_lr = lr.predict(X3_test)\n",
    "y_train_pred_lr = lr.predict(X3_train)\n",
    "\n",
    "\n",
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names3), \"coef\":list((lr.coef_.T))})\n",
    "fs.sort_values(by=['coef'],ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of LinearRegression on test is 1302.3568632529712\n",
      "The RMSE of LinearRegression on train is 1254.1831135610016\n"
     ]
    }
   ],
   "source": [
    "# 使用RMSE评价模型在测试集和训练集上的性能，并输出评估结果\n",
    "\n",
    "#测试集,输出RMSE结果\n",
    "#print ('The r2 score of LinearRegression on test is', r2_score(y_test, y_test_pred_lr)) #r2_score评价\n",
    "print ('The RMSE of LinearRegression on test is', np.sqrt(mean_squared_error(y_test, y_test_pred_lr))) \n",
    "\n",
    "#训练集，输出RMSE结果\n",
    "#print ('The r2 score of LinearRegression on train is', r2_score(y_train, y_train_pred_lr)) #r2_score评价\n",
    "print ('The RMSE of LinearRegression on train is', np.sqrt(mean_squared_error(y_train, y_train_pred_lr))) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeUAAAFsCAYAAADyj6FyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAdTElEQVR4nO3df5xVdb3v8ddHmMAjAqJgCBpYWuIvsNHgmkZRaWHiedxKu/7AK2oPra7V7Zd5y8nqnDx6++HVU8fbDzUtUapjpx4+jl6OHNJrGmNkKhqkcoM4gJg/sEzAz/1jL3Az7IGZYYb5zszr+Xjsx6z1XT/297v3rHnPd63vXjsyE0mS1Pt26+0KSJKkGkNZkqRCGMqSJBXCUJYkqRCGsiRJhTCUJUkqhKGsAS8iHo6I6b1dj94UEX8bEX+IiPURMWUXPu/6iDiwnWVnR8Td3fQ8T0bE27tjX1JPMpTVrzX6Y9z2j31mHpqZC3awnwkRkRExuIeq2tuuBD6cmcMy89dtF1Ztf6EK0ZUR8dWIGLSzT1o93+M7ux+pvzCUpQIUEPavAR7ewTpHZuYw4C3AqcA5PV4raYAxlDXg1femI+KYiFgUEc9FxOqI+Gq12sLq5zNVb3FaROwWEf8jIpZHxJqIuCEiRtTt96xq2bqI+Fyb52mJiHkRcWNEPAecXT33vRHxTESsioirI+JVdfvLiLgwIpZGxPMR8cWIeG21zXMRcUv9+m3a2LCuETEkItYDg4DfRMTvd/R6ZeYy4B5gct3+R0TEd6p6r4yIL23uSUfE6yLi3yPi2Yh4KiLmtmnT66rpvSPip1Vb7gdeW7feNmcqImJBRJxbTb82Iv6teq2fioibImJkO69Fe++x1OsMZWlr3wC+kZnDqYXCLVX58dXPkdUp13uBs6vHW4EDgWHA1QARMQn4R+B0YCwwAhjX5rlmAfOAkcBNwCbgY8A+wDRgBnBhm21OBN4ITAU+BVxbPcf+wGHAB9ppV8O6ZuZfq94v1HrCr228+Ssi4g3AccCyuuLrgY3A64ApwDuBc6tlXwTuAPYCxgP/q51dXwO8SO31OofO9cQD+HtgP+AQaq9HSzvrtvceS73OUNZA8M9V7/OZiHiGWli2ZwPwuojYJzPXZ+Yvt7Pu6cBXM/PxzFwPXAycVvXm3gv8S2benZkvAZ8H2t5o/t7M/OfMfDkz/5KZrZn5y8zcmJlPAv9E7VRxvcsz87nMfBh4CLijev5ngdupBWJn69pRD0TEC8ASYAHV6xgR+wLvAj6amS9k5hrga8Bp1XYbqJ0e3y8zX8zMbQZvVb3q/wx8vtrHQ9SCvkMyc1lm3ln9k7EW+CrbvnabdeY9lnYpQ1kDwSmZOXLzg217n/XmAAcDj0bEryLipO2sux+wvG5+OTAY2Lda9ofNCzLzz8C6Ntv/oX4mIg6OiJ9FxH9Up7T/jlqvud7quum/NJgfRmPbq2tHHVXt/1TgTcAeVflrgCZgVd0/Pv8EjKmWf4paT/b+qI10b9QDHl3Vp/41Wd5gvYYiYkxE3FydOn8OuJFtX7vNOvMeS7uUoSzVycylmfkBaoFyOTAvIvZg214uwB+pBdJmB1A7hbsaWEXtVC0AEbE7sHfbp2sz/03gUeCg6tTqZ6mFWXfYXl07LGtuAe6l1vuHWpD+Fdin7p+f4Zl5aLXNf2TmeZm5H/BB4B83X0eus7aqz/5t6rjZC9XPv6kre3Xd9N9Tez2PqF67M2jntdvOeyz1OkNZqhMRZ0TE6Mx8GXimKt5ELTRepnY9drMfAh+LiIkRMYxaz3ZuZm6kdq34PRHxn6rBV19gxwG7J/AcsL66bntBtzVs+3Xtiq8A50fEqzNzFbVrxv8zIoZXg8peGxFvAYiI90XE5n9Q/kQtPDfV7ywzNwE/Bloi4m+qa/Kz65avBVYCZ0TEoKq3XX/9e09gPbWBeOOAT7ZX8e28x1KvM5SlrZ0IPFyNSP4GcFp1HfTPwJeBe6pTtFOB7wLfpzYy+wlqg5Q+AlBd8/0IcDO1XvPzwBpqPcr2fAL4L9W6/xuYu511O6vdunZFZv4W+HdeCb+zgFcBj1AL3nnUBmwBHA3cV72mPwUuyswnGuz2w9ROj/8HcB3wvTbLz6uebx1wKPB/65Z9gdrp9WeBn1ML+PY0fI+332Jp14jMRmflJHWnqnf6DLVT040CSZLsKUs9JSLeU52K3YPaHbN+CzzZu7WSVDJDWeo5s6gNsPojcBC106SempLULk9fS5JUCHvKkiQVYpfeBH+fffbJCRMm7MqnlCSpKK2trU9l5uhGy3ZpKE+YMIFFixbtyqeUJKkoEdHu3eo8fS1JUiEMZUmSCmEoS5JUiF16TVmS1NiGDRtYsWIFL77oHT/7i6FDhzJ+/Hiampo6vI2hLEkFWLFiBXvuuScTJkwgoru+HEy9JTNZt24dK1asYOLEiR3eztPXklSAF198kb333ttA7icigr333rvTZz4MZUkqhIHcv3Tl/TSUJUkqhNeUJalALS27fn+DBg3i8MMPZ+PGjUycOJHvf//7jBw5stPPde655/Lxj3+cSZMmbVV+3XXXsWjRIq6++upO7xNg2LBhrF+/vkPrTp8+nSuvvJLm5uYtZYsWLeKGG27gqquu6tLz7wr2lCVJAOy+++4sXryYhx56iFGjRnHNNdd0aT/f/va3twnkEjQ3N/d4IG/atGmntjeUJUnbmDZtGitXrtwyf8UVV3D00UdzxBFHcOmllwLwwgsvMHPmTI488kgOO+ww5s6dC9R6qZtvqfy9732Pgw8+mLe85S3cc889W/Z39tlnM2/evC3zw4YNA2D9+vXMmDGDo446isMPP5zbbrttm7qtWrWK448/nsmTJ3PYYYfxi1/8okNtWrBgASeddBIALS0tnHPOOUyfPp0DDzxwq7C+8cYbOeaYY5g8eTIf/OAHtwTtBRdcQHNzM4ceeuiW1wBqt5C+7LLLePOb38ytt97aobq0x9PXkqStbNq0ifnz5zNnzhwA7rjjDpYuXcr9999PZnLyySezcOFC1q5dy3777cfPf/5zAJ599tmt9rNq1SouvfRSWltbGTFiBG9961uZMmXKdp976NCh/OQnP2H48OE89dRTTJ06lZNPPnmrQVM/+MEPOOGEE7jkkkvYtGkTf/7zn7vUzkcffZS77rqL559/nte//vVccMEFLFu2jLlz53LPPffQ1NTEhRdeyE033cRZZ53Fl7/8ZUaNGsWmTZuYMWMGDz74IEccccSWet99991dqkc9Q1mSBMBf/vIXJk+ezJNPPskb3/hG3vGOdwC1UL7jjju2BOr69etZunQpxx13HJ/4xCf49Kc/zUknncRxxx231f7uu+8+pk+fzujRtS9EOvXUU/nd73633TpkJp/97GdZuHAhu+22GytXrmT16tW8+tWv3rLO0UcfzTnnnMOGDRs45ZRTmDx5cpfaO3PmTIYMGcKQIUMYM2YMq1evZv78+bS2tnL00UdveU3GjBkDwC233MK1117Lxo0bWbVqFY888siWUD711FO7VIe2PH0tSQJeuaa8fPlyXnrppS3XlDOTiy++mMWLF7N48WKWLVvGnDlzOPjgg2ltbeXwww/n4osv5rLLLttmn+19LGjw4MG8/PLLW/b/0ksvAXDTTTexdu1aWltbWbx4Mfvuu+82n/U9/vjjWbhwIePGjePMM8/khhtu6FJ7hwwZsmV60KBBbNy4kcxk9uzZW9r62GOP0dLSwhNPPMGVV17J/PnzefDBB5k5c+ZW9dpjjz26VIe27ClrwOro6NbuHgUrlW7EiBFcddVVzJo1iwsuuIATTjiBz33uc5x++ukMGzaMlStX0tTUxMaNGxk1ahRnnHEGw4YN47rrrttqP29605u46KKLWLduHcOHD+fWW2/lyCOPBGrXYVtbW3n/+9/PbbfdxoYNG4DaKfAxY8bQ1NTEXXfdxfLl237L4fLlyxk3bhznnXceL7zwAg888ABnnXVWt7R9xowZzJo1i4997GOMGTOGp59+mueff57nnnuOPfbYgxEjRrB69Wpuv/12pk+f3i3PWc9QlqQC9fY/g1OmTOHII4/k5ptv5swzz2TJkiVMmzYNqA3KuvHGG1m2bBmf/OQn2W233WhqauKb3/zmVvsYO3YsLS0tTJs2jbFjx3LUUUdtGTR13nnnMWvWLI455hhmzJixpad5+umn8573vIfm5mYmT57MG97whm3qtmDBAq644gqampoYNmxYuz3lmTNnbrnv9LRp0/jQhz60w3ZPmjSJL33pS7zzne/k5ZdfpqmpiWuuuYapU6cyZcoUDj30UA488ECOPfbYjr+YnRCZ2SM7bqS5uTk3j8iTeps9ZZVkyZIlHHLIIb1dDXWzRu9rRLRmZnOj9b2mLElSIQxlSZIKYShLUiF25eVE9byuvJ+GsiQVYOjQoaxbt85g7ic2f5/y0KFDO7Wdo68lqQDjx49nxYoVrF27trerom4ydOhQxo8f36ltDGVJKkBTUxMTJ07s7Wqol3n6WpKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEIYypIkFcJQliSpEIayJEmFMJQlSSqEoSxJUiEMZUmSCtHhUI6IQRHx64j4WTU/MSLui4ilETE3Il7Vc9WUJKn/60xP+SJgSd385cDXMvMg4E/AnO6smCRJA02HQjkixgMzgW9X8wG8DZhXrXI9cEpPVFCSpIFicAfX+zrwKWDPan5v4JnM3FjNrwDGNdowIs4Hzgc44IADul5TqYNaWnq7BpLUNTvsKUfEScCazGytL26wajbaPjOvzczmzGwePXp0F6spSVL/15Ge8rHAyRHxbmAoMJxaz3lkRAyuesvjgT/2XDUlSer/dthTzsyLM3N8Zk4ATgP+LTNPB+4C3lutNhu4rcdqKUnSALAzn1P+NPDxiFhG7Rrzd7qnSpIkDUwdHegFQGYuABZU048Dx3R/lSRJGpi8o5ckSYUwlCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEIYypIkFcJQliSpEIayJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUCENZkqRCGMqSJBWiU9+nLEklaWnp3vWk3mZPWZKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEIYypIkFcJQliSpEIayJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUCENZkqRCGMqSJBXCUJYkqRCDe7sCksrU0tK960naMXvKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIK4Uei1Gf40RtJ/Z09ZUmSCmEoS5JUCENZkqRCGMqSJBXCUJYkqRCGsiRJhTCUJUkqhKEsSVIhDGVJkgphKEuSVAhDWZKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIXYYyhExNCLuj4jfRMTDEfGFqnxiRNwXEUsjYm5EvKrnqytJUv/VkZ7yX4G3ZeaRwGTgxIiYClwOfC0zDwL+BMzpuWpKktT/7TCUs2Z9NdtUPRJ4GzCvKr8eOKVHaihJ0gDRoWvKETEoIhYDa4A7gd8Dz2TmxmqVFcC4nqmiJEkDw+COrJSZm4DJETES+AlwSKPVGm0bEecD5wMccMABXaym+pqWlu5drzd1po59oT29xddR2rFOjb7OzGeABcBUYGREbA718cAf29nm2sxszszm0aNH70xdJUnq1zoy+np01UMmInYH3g4sAe4C3lutNhu4racqKUnSQNCR09djgesjYhC1EL8lM38WEY8AN0fEl4BfA9/pwXpKktTv7TCUM/NBYEqD8seBY3qiUpIkDUTe0UuSpEIYypIkFcJQliSpEIayJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUiA59IYW0mV8UIEk9x56yJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUCENZkqRCGMqSJBXCUJYkqRCGsiRJhTCUJUkqhKEsSVIhDGVJkgphKEuSVAi/JUpSv9fRbzfzW9DU2+wpS5JUCENZkqRCGMqSJBXCUJYkqRCGsiRJhTCUJUkqhB+JUq/yIyiS9Ap7ypIkFcJQliSpEIayJEmFMJQlSSqEoSxJUiEMZUmSCuFHoiTtlJ74WJsfldNAZU9ZkqRCGMqSJBXCUJYkqRCGsiRJhTCUJUkqhKEsSVIhDGVJkgphKEuSVAhDWZKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEIYypIkFcJQliSpEIayJEmFGNzbFZD6k5aW7l1P0sBiT1mSpEIYypIkFWKHoRwR+0fEXRGxJCIejoiLqvJREXFnRCytfu7V89WVJKn/6khPeSPw3zPzEGAq8KGImAR8BpifmQcB86t5SZLURTsM5cxclZkPVNPPA0uAccAs4PpqteuBU3qqkpIkDQSduqYcEROAKcB9wL6ZuQpqwQ2MaWeb8yNiUUQsWrt27c7VVpKkfqzDoRwRw4AfAR/NzOc6ul1mXpuZzZnZPHr06K7UUZKkAaFDoRwRTdQC+abM/HFVvDoixlbLxwJreqaKkiQNDB0ZfR3Ad4AlmfnVukU/BWZX07OB27q/epIkDRwduaPXscCZwG8jYnFV9lngK8AtETEH+H/A+3qmipIkDQw7DOXMvBuIdhbP6N7qSJI0cHlHL0mSCuEXUkj9gF9wIfUP9pQlSSqEoSxJUiEMZUmSCmEoS5JUCENZkqRCGMqSJBXCj0TJj9NIUiHsKUuSVAhDWZKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEIYypIkFcJQliSpEIayJEmF8FuipIL5DV7SwGJPWZKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEIYypIkFcJQliSpEIayJEmFMJQlSSqE3xIl9QK//UlSI/aUJUkqhKEsSVIhDGVJkgphKEuSVAhDWZKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIKYShLklQIv5BCkiqd+aIQv1REPcGesiRJhTCUJUkqhKEsSVIhDGVJkgphKEuSVAhDWZKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKsQOQzkivhsRayLiobqyURFxZ0QsrX7u1bPVlCSp/+tIT/k64MQ2ZZ8B5mfmQcD8al6SJO2EHYZyZi4Enm5TPAu4vpq+Hjilm+slSdKA09Vryvtm5iqA6ueY9laMiPMjYlFELFq7dm0Xn06SpP6vxwd6Zea1mdmcmc2jR4/u6aeTJKnP6moor46IsQDVzzXdVyVJkgamrobyT4HZ1fRs4LbuqY4kSQNXRz4S9UPgXuD1EbEiIuYAXwHeERFLgXdU85IkaScM3tEKmfmBdhbN6Oa6SJI0oHlHL0mSCmEoS5JUCENZkqRCGMqSJBXCUJYkqRCGsiRJhTCUJUkqhKEsSVIhdnjzEJWnpaV715PUeR6H6gn2lCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEIYypIkFcJQliSpEIayJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUCENZkqRCRGbusidrbm7ORYsW7bLn62v8Nhlp4PL4HzgiojUzmxsts6csSVIhDGVJkgphKEuSVAhDWZKkQhjKkiQVYnBvV0CS1DOjrx3R3ffYU5YkqRCGsiRJhTCUJUkqhKEsSVIhDGVJkgphKEuSVAhDWZKkQhjKkiQVwlCWJKkQhrIkSYUwlCVJKoShLElSIQxlSZIK4bdE9TC/pUVS6Tr6d8q/Zz3PnrIkSYUwlCVJKoShLElSIQxlSZIKYShLklQIQ1mSpEL06Y9EOTxfktRVJX4UzJ6yJEmFMJQlSSqEoSxJUiEMZUmSCmEoS5JUCENZkqRCGMqSJBXCUJYkqRCGsiRJhTCUJUkqxE6FckScGBGPRcSyiPhMd1VKkqSBqMuhHBGDgGuAdwGTgA9ExKTuqpgkSQPNzvSUjwGWZebjmfkScDMwq3uqJUnSwBOZ2bUNI94LnJiZ51bzZwJvyswPt1nvfOD8avb1wGNdr2632Qd4qrcr0YNsX9/Vn9sGtq+vs33d4zWZObrRgp356sZoULZNwmfmtcC1O/E83S4iFmVmc2/Xo6fYvr6rP7cNbF9fZ/t63s6cvl4B7F83Px74485VR5KkgWtnQvlXwEERMTEiXgWcBvy0e6olSdLA0+XT15m5MSI+DPwrMAj4bmY+3G0161lFnU7vAbav7+rPbQPb19fZvh7W5YFekiSpe3lHL0mSCmEoS5JUiH4XyhHxxYh4MCIWR8QdEbFfVR4RcVV1S9AHI+Koum1mR8TS6jG7rvyNEfHbapurIqLRx8B2qYi4IiIerdrwk4gYWbfs4qquj0XECXXlDW+HWg3Su69q99xqwF6vioj3RcTDEfFyRDS3Wdbn27c9ffW2tRHx3YhYExEP1ZWNiog7q9f+zojYqyrv9HHYmyJi/4i4KyKWVL+XF1Xl/aV9QyPi/oj4TdW+L1TlDY+diBhSzS+rlk+o21fD47MEETEoIn4dET+r5sttX2b2qwcwvG76vwHfqqbfDdxO7fPVU4H7qvJRwOPVz72q6b2qZfcD06ptbgfeVUD73gkMrqYvBy6vpicBvwGGABOB31MbgDeomj4QeFW1zqRqm1uA06rpbwEXFNC+Q6jdZGYB0FxX3i/at512t9uO0h/A8cBRwEN1Zf8AfKaa/kzd72mnj8NebttY4Khqek/gd9XvYn9pXwDDqukm4L6q3g2PHeBCXvmbehowt5pueHz2dvvq2vlx4AfAz6r5YtvX73rKmflc3ewevHJDk1nADVnzS2BkRIwFTgDuzMynM/NPwJ3AidWy4Zl5b9belRuAU3ZdSxrLzDsyc2M1+0tqnw+HWvtuzsy/ZuYTwDJqt0JteDvUqtf/NmBetf31lNG+JZnZ6K5v/aJ929Fnb1ubmQuBp9sUz6L2msPWr32njsOer/32ZeaqzHygmn4eWAKMo/+0LzNzfTXbVD2S9o+d+nbPA2ZUx1p7x2evi4jxwEzg29X89v429Hr7+l0oA0TElyPiD8DpwOer4nHAH+pWW1GVba98RYPykpxD7b9y6Hz79gaeqQv4EttXb6C2r6/aNzNXQS3YgDFVeWffx2JUpzKnUOtN9pv2Vad2FwNrqP2z8HvaP3a2tKNa/iy1Y63Y9gFfBz4FvFzNb+9vQ6+3r0+GckT8n4h4qMFjFkBmXpKZ+wM3AZvvxd3ebUE7W97jdtS+ap1LgI3U2sh26tsn29doswZlRbavi/pafbuqT75fETEM+BHw0TZn47ZZtUFZ0e3LzE2ZOZnaWbdjqF1C2ma16mefal9EnASsyczW+uIGqxbTvp2593Wvycy3d3DVHwA/By6l/duCrgCmtylfUJWPb7B+j9tR+6pBIicBM6pT67D92542Kn+K2qm1wdV/hMW0rx19pn1d1N9uW7s6IsZm5qrq9O2aqryzx2Gvi4gmaoF8U2b+uCruN+3bLDOfiYgF1K4pt3fsbG7fiogYDIygdumi1N/fY4GTI+LdwFBgOLWec7nt64kL1b35AA6qm/4IMK+ansnWAzDur8pHAU9QG3yxVzU9qlr2q2rdzQO93l1A+04EHgFGtyk/lK0HIjxObfDQ4Gp6Iq8MIDq02uZWth7scGFvt6+uPQvYeqBXv2pfg/a2246+8AAmsPVAryvYeiDUP1TTnT4Oe7ldQW08ydfblPeX9o0GRlbTuwO/oPYPf8NjB/gQWw+EuqWabnh89nb72rR1Oq8M9Cq2fb3+QvXAC/8j4CHgQeBfgHFVeQDXULte8lu2/oN/DrUL98uA/1pX3lzt6/fA1VR3QOvl9i2jdm1jcfX4Vt2yS6q6PkbdSHFqI0J/Vy27pK78QGojzJdVv6RDCmjf31L7r/SvwGrgX/tT+3bQ9obtKP0B/BBYBWyo3rs51K7DzQeWVj83/6Pb6eOwl9v2ZmqnKR+sO+be3Y/adwTw66p9DwGfr8obHjvUepu3VuX3AwfW7avh8VnKg61Dudj2eZtNSZIK0ScHekmS1B8ZypIkFcJQliSpEIayJEmFMJQlSSqEoSxJUiEMZUmSCvH/AViBFayk6scFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#在训练集上观察预测残差的分布，看是否符合模型假设：噪声为0均值的高斯噪声\n",
    "f, ax = plt.subplots(figsize=(7, 5)) \n",
    "f.tight_layout() \n",
    "ax.hist(y_train - y_train_pred_lr, bins=40, label='Residuals Linear', color='b', alpha=.5); \n",
    "ax.set_title(\"Histogram of Residuals\") \n",
    "ax.legend(loc='best');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAADQCAYAAADcQn7hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO2dfZxcdXnov8/OziazUbMbjBaWhABywwUxCW4xNve2QlsiohBRCVRv8aVSrb1F5JOatHwEWlqjUUF7vSqVcmmlGN668qYrAt7e0iaQuAkxwtbwlmRDSzTZqMlAJrvP/eOcszk7e15+83JmZnef7+czn535zZlzfnN2znN+z7uoKoZhGFnQ1uwJGIYxdTEBYxhGZpiAMQwjM0zAGIaRGSZgDMPIjPZmTyALXvva1+qCBQuaPQ3DmDZs3rz5Z6o6t3x8SgqYBQsWsGnTpmZPwzCmDSLyQtS4qUiGYWSGCRjDMDLDBIxhGJkxJW0whhFF38AQ6/oH2TNc5LiuAquWL2TFkp5mT2tKYwLGyBzXCztLAdA3MMSae7ZRLI0AMDRcZM092wBMyGSIqUhGpgQX9tBwEeXohd03MFTVdtWyrn9wTLgEFEsjrOsfrMv+jWhsBWNkStKFHV45uG5XLXuGixWNtxq1rO6aqRqagDEyxfXCjttuaLjIiasfqPnCOK6rwFDEMY7rKlS1v0YSpd6tunMr1923neFDpVS1s5mqoalIRqbEXcDl40kXeppqtWztI5y4+gGWrX0kVqVatXwhhXxu3Fghn2PV8oUTtnXdZ6OIWt2VRpX9h0qp6mSzVUMTMEamuF7YUduVU35hVGK3WbGkh89edAY9XQUE6Okq8NmLzphwF4/b59V925omdFzUuDih0WzV0FQkI1OCCzjNBlC+XVydxfCFUandZsWSnlS1IG6ft23YOTanRqoZfQNDtIkw4lB5MkpoNFs1NAFjOFOtsdDlwi7fbtnaR1IvjCzuznGfLb+862mAjjuvwWrKRbhAtNBYtXzhOBsMxKuGWWAqkuFE1m7kclxUK1f7TiVU8tl6qBlJ5zVqNRWQz8m41wKcfeqEZGZn1TArbAVjpNI3MMRVd2ydcCet5128nDTVqm9giIOvHJnwuXIhVOmqK+qOL0xcwUB91Iw4ley6+7az/1Ap9nNnLejmX5/ZNzYvBW7bsBOA61ecMW5b1xVkFpiAMRJJW6Y3Mo5k0wv7WNc/GKk6AXTm25iRb+PK9VtY1z/I2afO5e7NQxW5aKMEW/l+oH5qRtz5SxIuAI89s2/CWCBkek+Y0zLRyTIV25b09vaq1YOpD3G2kICergKPrT6n7sctj9+oJ9XMOatgtbTzWw3dnXkGPnNuRZ+p9fuJyGZV7Z0wbgLGSOLE1Q/EenQK+VxN+nzSjzqLCy9AgOfWnl/RfCrZphKyEqSFfBsvl0ad5hg1h0r/t3ECxlQkI5E4N2dOpGbhkhRhmqXqFWU7cYl4rTQqtlwYnX3qXB59em+kcApvVw/BWiyNOs0xOHZWaRrmRTISifPmfPHiRTX9+NIiTLOM04iynbhEvFYSFRvlHfrWhp2R3qIVS3p4bPU5PLf2fB5bfQ5dhXxdvmd4jlfdsTXW45dlMJ4JGCORat2caeH2aT9ql8jeaujuzEfO3eUiS8qXKifJxRxQLI3wZ/c8OW6sb2AIkYnbtgnk2yLecGRENTasIE6Yt4nUHLlsKpKRSqVuThdVIk4VmF3Is2ztI+wZLtLVmWdGexvDxWSPiiuFfI7z33Ts2P7DaopLxGvcNgJjK5EA17v/odIoV/dt4/oVZyTaY0YV2mtcDsSpPVGueWDMc1hL5LKtYIy646JKRK1Q8m3CwcNHxtSI/YdK/DIi1qUaciK858093L15KFJNcQnsW7V8IVFrCIUJalIlKt7tG3cB6auewyO1O2SiBF/5KjUXsYSqNkHSBIxRd1zUjSjV61Uz2ymVXUQjo+kXVU/KxRzYjB59em+k4Lvqjq1cuX4LM9rb6O7Mx6qCK5b0OOVIgRdV66rQBCuFRsQUtYnEJoMGdqDROsY8ZaoiiUgX8E3gjXiC/sPAILAeWAA8D1ysqvtFRIAvA+8ADgEfVNUf+fu5DLja3+31qnprlvM2asM1wa5c9Tpx9QNVHW/fwVcS35/R7gXfxQmH4AIfLpYQ4P1L50+Ihg3oiflugb0i8Batf3xX7PGiOHnNg3R25Dh4uP5xP2ECWwzEqzv1TJDMegXzZeB7qnoqsAh4ClgNPKyqpwAP+68BzgNO8R+XA18DEJE5wDXAW4CzgGtEpDvjeRs1UEntlTCzq/SeBC7ZOIaLJeeLXYFvbdjJ4uu+T9/AEFf3bePkNQ+yYPUD3t9jCpHG5xHVcd6iksPKq/zzBw+PkKvBkAuM80B15tsiDcZp6k61/78oMlvBiMhrgN8EPgigqoeBwyJyIfA2f7NbgR8CnwYuBP5evci/DSLSJSLH+ts+pKr7/P0+BLwduD2ruRvuJAWeVRqQFnUxNIvhYomr7tw6TkUbUY0M0a8no6NKm3hG3Wp45chRYXsoQfAODRcjjd3gXmLDhSxVpJOAvcAtIrII2AxcAbxeVV8EUNUXReR1/vY9wK7Q53f7Y3Hj4xCRy/FWPsyfP7++38SIJM1bVGnwWavFlLvYf+qNAjdcvJhPrt9S8WdFqCgiOFCDorxE9UqQzFJFagfOBL6mqkuAgxxVh6KIM9DHjY8fUL1JVXtVtXfu3Ilp60b9qbQcY1rwmeGx6YXKV0n5NqGWrJ+symhmKWB2A7tVdaP/+i48gfOfvuqD//el0PbzQp8/HtiTMG40mUojQF2Cz7Im3ybjPEX5FvSjfssvu1AJK8+al+pNSyMLL1Zmp1dV/wPYJSKBZei3gZ8A9wKX+WOXAd/xn98L/L54LAUO+KpUP3CuiHT7xt1z/TGjyVRa8KnZLUJyIpRGlc6Odm5YuZjHVp/DuvctrilCNqDWi7tWHn16b83Rz1mkZ2Qdyfs/gdtEpAN4FvgQnlC7Q0Q+AuwE3udv+yCei3oHnpv6QwCquk9E/hJ4wt/uLwKDr9F4wjaUQsztP6qyGsS7PxtFUmRqVEEtgFkdOV4ujSaWrVx28hxu++hbWVClm70e7BkuTjDOdnXm+dXLR5w8WlmV0bRyDUYiYYHi+oPNiTCqOsGIO7uQr1vYfz0I14WJK0tRXtrh/X/7b+M8Sae8bhaHDo823UgdV+Pm6r5tkSpXm8BrZuY5UEzuq+SKlWswIolyMwNjlePC5SLTqqwFhFcK4R93KwkXGK+ypQWXhc9TT+g8ZVUUqxLyOeHsU+dGup0ffXpv5GdmFyovSlUNtoKZxkQl1+VzAkrFgWKtgABtbTLOvSzAzHxbZDCe+NIzrixmENXbe8KcyIJM9UzErJbuzjznv+nYyJKen73ojMQI5p6uQt0KZ1lFO2MCWVaNawbLTp7DT1785dhKK1h9dXfmOXCoRFK8byGf48z5s8cV0g7GZ+bbnFdvjSRQ3+L+j4HhOS4DvPx71lJALE7AtKCTzqiFStqe1sOrE3b7RmXhxlEHx80EHntm3zhBEFbtkpMJvDiQDc/uj+x/1IrCBY6mVsTdJIaGi5GepaguCZMxDsZoMJX2LqrGLZnPCV2Fo3Ek6963iIHPnMtza89n6UnuKWKj6t01Wyg7wLnBWUB3Z97JLZzVRRbI8zjBnhOJzFp3zQivB2bknUJUWls1qtBQlA0muOP1pOjqG57dX9F8m20cLSfn2KI14Pw3HUvvCXNiXdxw9Jwlhf7nczKhTIUL+w+V6BsYij12MF4e9u/SNbNemICZQlQaWRuX1BY1lpZX1NWZr3gF0EoU8rmxglSugm/9E7voPWEOX7x4UWpV/rh+TrM6chw+kqbAxbPmnm10d+Yj1bi44L9GtpM1ATOFqKaOR1xSW1q7jnKvSzPtFCJUnIfTVcgj4s07J0KxNMKjT++NNPTGURpR1vUPjsWfJAnlVcsXTsjOBpzrv8R1lyyWRpjR3kYhn3MWGPXMlk6dt3mRpg716G/jut+4H3wzqLS8QRCUVq+eRN2dea551+mpvYeqyZAGz5C+8qx5sTlKAtywcnFDBEYcFmg3DcjqzhRl26lEuAS2je7OPMOH3Is/uVJpyE6gMtYr+XL/oRKr7toKjO+hFP4/RPXRduVVM9u5fsUZPPr03tgVajP7TydhKxgjlaTujq64BH41iq5Cnlkz2jOJAeoq5HnnoomBb7UQxLtktUKtBxYHY1RNPbwLgTcry4ZqLrQJY50LXCnkc3xg6XynhmjDxRLf2rCzrh6y4JxV26OqmdgKxoglWOaX5yRVSxB6X029k3rQ3ekJiEoM0uWu+ZPXPNhQb1lgX2llIQI12GBE5PXAXwPHqep5InIa8FZVvTmDeRotQvlyPCgtGMTDvPSLIim1tifQ2ZEb6wHUKLoKebZcczSpz7VzQaB6gGeruXL9Fo7rKtQkXLo783R2tFcksJXKm521Ei5G3v8D3AL8uf/63/HajpiAmYSEVyVh70tXIc+1F5w+zlAcZdh1CRyLI+uWHFGUB7l2xcSMlHPm/Nlce+/2ccmMtdpswp6mKCNwVOJkswtZ1YqLgHmtqt4hImsAVPWIiLRWCKbhRPmqJOx9GS6WWHXnVja9sC/WWwHeRXbVHVsbMd26UC5MXBcgWXQPKG/iFn4dZ8DNIvitkbgImIMicgz+ii4oZ5nprIyaiGslkuaWLY0qt23Ymbp0T1MTWilGBmDB6gfo7syj2no1aQIaGfzWSFwEzKfw6uWeLCKPAXOB92Y6K6NqklqJuCSz1UMwNEq4BLVQklZcAc3OiHbxQLVqLEstpAoYVf2RiPwWsBDv5jSoqq15GzASEx6bXRO3VsKlOMN397BdqRXJtwnXXnB6s6fRFFy8SJ8AblPV7f7rbhG5VFX/d+azMyomKeHxhpWL61riMXzBv3igWHU3QhfyOWHdexdF5kjVu2xlrk1oo7qqfoERPErVSeqCOVVxUZE+qqpfDV74jeo/CpiAaUGSEh6DH/N1922PVBmWnTyHH+084HSxCnDpW+aNNYnPsqJ+Uq5PvXstCXDpWfNSyzBEERhlo1SdtC6YUxUXAdMmIuL3jEZEckBHttMyquXsU+dGBrIFrUSCH3/c3TQ8nnRpKXD35iF6T5jDiiU9FddSKafaKvf1LpKkwO0bd3Hbhp3MLuQ5ePiIU62WnEhiVG2ltXqmCi4Cph+vj9HX8c7/x4DvZTqraUS9l81xVeTLx8N32WAOQTBZMIe0mr3hC+TSt8Rn+7qQ8+0UlX73uBVbT1eB/QdfSWwAH0cgKMs9TnF1VwBGVRPnXmmtnqmCi4D5NPCHwMfxVpDfB76Z5aSmC1ksmyv9IUfNYdVdW8eCzNJczkPDxTFBVIt7OqitAqQGAoaF8syY5m8LjinU/eJ9uTQaK2TScqyqqdUzFXDxIo0CX/MfRh3JYtlc6Q85ag6lER27e4dTBOIIjlerjTcQsGmBgOFM5ah2JIBz0ahKKJZGeDnC3uMSENfIKnKtRGw2tYjc4f/dJiJPlj8aN8WpSxbL5rj+xIcOH4ks/u0aG9NVyNelh3MSQWW5OEqjyj9udMtUzsqhVb7f7s68U0bzZMyErgdJK5gr/L/vbMREpiNZLJuDH2x5Hs3+Q6VI9cs1Nma4WMqk1UhAecnHOFqtH1xnR7uzkJiKgXRpxK5gVPVF32N0s6q+UP5o4BynLFGrjXosm1cs6WHWjIn3jmJphOvu2z6ub9LZp851ar0B2V3cIozd3RtBULahHkx1I22tJBacUtUR4JCIzK72ACKSE5EBEbnff32iiGwUkZ+KyHoR6fDHZ/ivd/jvLwjtY40/Pigiy6udS6uR5bI57oe//1BpXN+kuzcP8Z4394zNobsz39AqZPk24YaLvXonceqdyz5ibL2RJKUNhPs+uQiiqW6krRUXL9LLwDYReQg4GAyq6p84HuMK4CngNf7rzwE3qOq3fdf3R/AMyB8B9qvqG0TkEn+7lX79mUuA04HjgB+IyH/xhV9L4+KCzmrZ7Kr6BNX0g8r4fQNDrLprK6NV9OmplJwI6953NDo3rcVHQFDyMtzh4LY6FbEqjSizZrSP1ZBJctVPByNtrbgImAf8R8WIyPHA+cBfAZ8SEQHOAX7P3+RW4Fo8AXOh/xzgLuB/+dtfCHxbVV8BnhORHcBZwL9VM6dG0ajIzTghFlx0LmIivNpZ1z9YVROwaogKzCuPz1l159ZxIfv5sniZvoEhrrpja12NuuHzEeX9gYn1c4xoXNzUt/pqzKl4RvRBVT3suP8bgT8FXu2/PgYYVtWgxPpuIPgP9QC7/GMeEZED/vY9wIbQPsOfaVkaEbkZJ8Tu3LRzgptWgEK+LTLwrE2EE1c/kFkyZJKbOxC6EN0AbkJv2dDr4PvXu4RlWO2JKqNw9qlzefTpvVy5fgvr+genRU5RtbgkO74D+AbwDN6/90QR+UNV/W7K594JvKSqm0XkbcFwxKaa8l7SZ8LHuxy4HGD+/PlJU2sIjYjcjBNiUcWSFOhoz6FMdAUHF2i9au+GCUoqxFXZL5ZGuPbe7bxyZDRitacTVlNBQJ5LfZtqyOeEg68cGRO45blF0zWnqFpcTGNfAs5W1bep6m8BZwM3OHxuGXCBiDwPfBtPNboR6BKRQLAdD+zxn+8G5gH4788G9oXHIz4zhqrepKq9qto7d+5ch+llS5zxr55GwUqF1YFiaZxROappepxEdyX4bE9XgRtXLmbgM+dy/YozxurbRjFcLEUKyrgguuB719uD092ZB78oVWAEX3PPtnHxQ0krU2MiLgLmJVXdEXr9LPBS2odUdY2qHq+qC/CMtI+o6vuBRzlasOoy4Dv+83v91/jvP+InWN4LXOJ7mU4ETgEed5h3U8nKBQ3eXXTJX3y/4pVGkFH92OpzeG7t+YzGqBa1rGCCur2PrT5nwh09SqBVQyCk6yGs823CjSsX8/za8+nsaJ9QoqFceLRiTlHfwNC40IOogMpm4SJgtovIgyLyQRG5DLgPeEJELhKRi6o45qfxDL478GwsQfHwm4Fj/PFPAasB/Do0dwA/wUuy/MRk8CBl5YIOvDxxrtakS/jgK+OjeeMu0J6uQk0xKeUXW5KtpJDPTSjMnUYgpKt1aweHy4lQGvVUrr6BISfh0YiVaSUE5zYcelC+6momqX2RROSWhLdVVT9c3ynVzmTrixSuyBaUPSjvxxOQluGcVtMlnxNmdbRzoFhidiHPL14uTQig+8DS+fSeMKfqQk7BCiZtzjkRvnjxooo6FHR35hn4zNE2JFf3beP2jbsqNvSW25oK+Rwz822Rgjv8fVqtu2LcuS3/H2RN1Z0dVfVDCY+WEy6TjfAdCMYbXKPuRGlLcc/Aq7FBYkEio+LZGqKic9c/7vUuqia6NkoNjJtzUOKgkmOoMnZO+gaGuHvzUFVepPJPFEsjqJKq1rZaTlErqmxhrLNjk0lbkbiuBsrJt0lVJR+jjpvUmzpoJpYUTJg05x7f7VtpL+dqujSmEXRRnExlLVt9BeMSaGdkSNqdpvz9VcsXsuquranBcKVRranKXLndIepHLDChlGVgcAxfoHHBajA+XcGlO0BALYIlzhUfGMFbWaCU0+plIBqZdmJEkGYcnF3Ij/MSrOsfZOWvz6PTIflmRLVql3N4XlHG1KDPdFTzsHKDI3jqVlzrjnC6wvNrz+fGlYvHVJB6kRMZU2nev3R+Zh6+RtNqKls5sSqSiHwq6YOq+qVMZlQHJpOKlFYVv028cpLhFUshn+M9b+7h/q0vZtJILN82PkcomGea6pC2XE9SlQR4bu35E8YXX/f9ir9juBIeRBthp2OF/yypRkUKwvsXAr+OF48C8C7gn+s7velLWoLfqDIh8bBYGnHOM0qiq5DnnYuO5YEnXxxTOeJybFxUhzh1b2i4mGjHgeiVXN/AEAcPH4nY+iid+TaKpdFx+86JMLvQzvCh+ALik00VmqzEChhVvQ5ARL4PnKmqv/RfXwvc2ZDZTROCH3vaRRimGuES2GSCv7NmtNN7wpyx1iNxhO/2XX4L1qjq/0m5TGnzjVJPXBIvo3KrSqNKZ0f7OHe20RxcjLzzgXBy42FgQSazmebUmmwoKfsYUR1XOc4lj+bqvm3jVkth42rw+U0v7Bsz0FaTy5SLKZVXi6u1Vdy00x0XAfMPwOMi8k94v513A3+f6aymKUneFhcU2Hfwldj3o2reJmV49w0Mpapi5eqaS5HwckZGlevu2z7BJlKLwLVCUK2BS7mGvxKR7wL/3R/6kKoOZDut6UGUofGzF50xZo+JulC7O/P86uUSce1+4hIEhej6KxB/t1/XP+gkKMq3CV5X4ibff6g0tjoKWqe0R6xsXGr3TlaP0FTE1U3dCfxCVb8M7PaTDo0YXJLPkly6j60+h56uQuTF7SXkVT6nIAkxiri7fa21YWqp01Ia0QnCMqjgn5Q02WpuWldaOWGxFlzqwVwD9OJ5k24B8sC38MoxGGW41guJS/u/9t7tiSUja7nooyJm4+72rfgDDyr4b3phX2QXyQ8snZ9qsHah0S7sqVxjxmUF827gAvx6vKq6h6MubKMM13ohcWrJcLGUSVU58NrHugZlNaq+SSWtUIJzdv2KM/jA0vljK5mcSF2FS6Ozk6dyjRkXI+9hVVURUQARmZXxnFqatLuba/JZVuUpk9gzXHSO/6jECxPkI1XzfSpJlwqrctevOKMuAqWcZjSpb/WExVpwWcHcISLfwKtE91HgB0zT3tQudzfXeiHV1jKphUo8K10V9A5SjY5jqYXylU2jDLfNuNhbrcZMPXEp1/AFvCr/d+PZYT6jql/JemKtiMtSNq2SXWDMu3L9Fma0t9W1CVgalVygldhnh4te18h6dn58zcx8U/JrmnGxZ1n9sNm4GHk/p6qfBh6KGJtWuNzdoqrQB2pUuTFvuFiikM/R3Zmva9mBKLo784kXaLnqV2n+T72Lbx8olsZ6EzWSZmQnJ/1mJjsuNpjfxStzGea8iLEpj2sv6Tg7R9wKaEZ7W8XBad1+yL6LICjkc1zzrtNj7UdRXox6dxeolGapB8262KdqblSsgBGRjwN/BJwsIk+G3no18K9ZT6wVqeXu1jcwFGsErSYjOijy1JNiLA5KbwKxrtAowVdNRG5XIc8Bv1peFAU/MTGNZqsHU/VibwZJ5RpmA93AZ/ELcPv8UlUnNt5pIbIs11BNjERaSYasmNWRo6uzgz3DRdpiomp7ugrs8Y3WUaQJsICgJMKmF/ZNSC8I3ouL73Gpime0NnHlGlyKfi8FtoeyqV8NnKaqGzOZaR1oZj2YKAF03X3bM7exlJNrE9ogtWxmWoIkTKyvEsWNKxePa04WJYTjssXjasEYk4daSmZ+DTgz9PpgxJhBdERmeW9lV2Z15Mjn2qpSn3IivHpGu9NnAwGQtMJKm3658yhOxXC1YRlTB5c4GNHQMkdVR5kCtXyzyP2IsmVUW3j75dIow8VSVWUjR1U54CiYFhxTGFd2sRoUt8jfqeyONaJxETDPisifiEjef1yB191x0pJVOHg9g7ECe0k14um4roLzquCxZ/Zxdd+2sY6P1YayuHz3Vq8fa9Qfl5XIx4CvAFfj/d4fxm8yP1nJKhy8GeH/5QTN24PVj4uAun3jrrGw+2q/g6tAMw/N9MIlkvclVb1EVV+nqq9X1d9T1dTe1K1MVuHgzQj/DxNu3g7jG9knqT8jqmOrt1XLF5KvMCTX1BwjjqQ4mD9V1c+LyN8QcSNU1T/JdGYZUk9jY3m9Wpc1g4tXplIELzam3FsVbkZ/8poHY2u0lJcHuPKOLbHpAvk24VUzjxbVPvvUuazrH+TK9VvMzWyMI2kF85T/dxOwOeIxaamXsbHclrP/UMkpkKzewgU84Zi2Mrv0LfNiPx/OqVqxpCdRTq573yIGPnMuz609n1XLF3L35qGWbb5uNJdYAaOq9/l/b416NG6K9adexsYoW04zCIRjWqJe7wlzEhu2hVd1cfvq8bsfBkzlWiZG7SSpSPeRcB9T1QuSdiwi8/CKg/8aMArcpKpfFpE5wHq8zgTPAxer6n4REeDLwDuAQ8AHVfVH/r4uwzMyA1xfDwFXD2NjK9TrCAp5r+sfTKxYdzRGJ36FFS5F6ZoWMZVrmRi1k6QifQH4IvAcUAT+1n/8Cvixw76PAFep6n8FlgKfEJHT8NIOHlbVU/A8UkEawnnAKf7jcrxgPnyBdA3wFuAs4BoR6a7gO2aGi80m3LK0XqUZggpuhXxuzKYyNFzkWxt2jhMIORHe82ZPkLqstsL2GddV3lSuZWLUTlLjtf8LICJ/qaq/GXrrPhFJ7eyoqi8CL/rPfykiTwE9wIXA2/zNbgV+iJeZfSHw935Q3wYR6RKRY/1tHwryn0TkIeDtwO3uXzMbXNqMvHpmO9decDoAf3bPk7HbVcIXL17kLDDu3jxE7wlznFYU5Z4ml1VeqzdfN5qLSxzMXBE5SVWfBfA7Csyt5CAisgBYAmwEXu8LH1T1RRF5nb9ZD7Ar9LHd/ljcePkxLsePz5k/f34l06ua8tT+rs48r5RGxnUbHC6WWHXnVhBSuxS6MKsjx4olPVy5fovT9oH6lBbfUq1QmMq1TIzacREwVwI/FJEgencB8IeuBxCRV+FVw/ukqv5C4ltORL2hCePjB1RvAm4CL9nRdX61Un6XX7b2EQ6VXcjVpgtEkc95Wm0lAXF7hovcsHLxhJVGEIgX1Ja5cv0W1vUPViwgLHjOiMMl0O57eHaRK/zHQlXtd9m5iOTxhMttqnqPP/yfvuqD/zcI2tsNhP2oxwN7EsZbkqyNm0GOUSVBfcd1FSJtKjesXMyNKxeP5T2Zm9moNy4lMzuBTwEnqOpHReQUEVmoqvenfE6Am4GnVPVLobfuBS4D1vp/vxMa/2MR+TaeQfeAr0L1A38dMuyeC6xx/4qNJet0gcB4GlZNXFWfqJXGsrWPNLyKvjF9cEl2vAWv4f1b/de7gesdPrcM+B/AOSKyxX+8A0+w/K6I/BSvHOdaf/sH8ZIod+B5q/4IwDfu/iXwhP/4i1YueFWvdAGBCSH75Yj1SPAAAAqxSURBVHaSFUt6Eo/nEt9jbmYjS1xsMCer6koRuRRAVYuSYEgJUNV/Idp+AvDbEdsr8ImYff0d8HcOc20ISbVtr7tve6p3p7szzzXv8jxLUfVi8m2w7n2LgXTjaZw3KUgPSMNqtBhZ4tR4TUQK+IZVETkZeCXTWbUwcW0+N72wj/VP7HLyFAUtUPsGhiaI4HxOWPfeRWOCJE1NqXUFYm5mI0tcBMw1wPeAeSJyG57q88EsJ9XKxIXG375xl3Oz9+DiX9c/OEEglUa0IvtHrSuQVnMzN7ovtJEtiQLGV4WeBi7Ci8YV4ApV/VkD5taSxK0MXIULHL3462H/qMcKpFXczFO5Cfx0JdHI69tF+lT156r6gKreP52FC9Rumwhf/PUIs59KVeIscXLq4aIibRCRX1fVJzKfzSRg1fKFFRfy7vT7AZUv+etl/2iVFUitmEdr6uEiYM4GPiYiz+N1FBC8xc2bspxYq7JiSU/FbUi6Z83gJxEenVazfzQb82hNPVwEzHmZz2KSMVxhj6OkO/BUWX3UA/NoTT2S6sHMxCv4/QZgG3Czqh5p1MRamUqjde0O7Iat6KYeSSuYW4ES8P/wVjGn4eUiTXviyjTM6shx+MjoOPuM3YErw1Z0U4skAXOaqp4BICI3A483ZkqtT9Kd1uI4DOMoSQJmzNCgqkccsgOmFXF32nLhEy6kbcLHmG4kCZhFIvIL/7kABf914EV6Teaza0HShERSKkG4Xm44iAzM7mBMTUQriECdLPT29uqmTZvqvt9y4QGejSUc2LZs7SORBuCcSGS0b1chzytHRhP3aRitjohsVtXe8nGXcg2Gj0ukaaWpBMPFkkWvGlMWlzgYw8cl0jTOhR23gqnkWGbDMSYbtoKpAJfcobiukZe+ZV7keFwrk/JjlXeRtNKWxmTABEwFuLScjUs+vH7FGZHj17zrdKc2tpYIaExGTEWqANdI0yQXdpxKk7ZPSwQ0JiMmYCoki0hTl31aIqAxGTEVaZLgop4ZRqthK5hJgiUCGpMREzCTCEsENCYbpiIZhpEZJmAMw8gMEzCGYWSGCRjDMDLDBIxhGJlhAsYwjMwwAWMYRmZMGgEjIm8XkUER2SEiq5s9H8Mw0pkUAkZEcsBXOdrd4FIROa25szIMI41JIWCAs4Adqvqsqh4Gvg1cWMsODx8+zM9//vO6TM4wjGgmi4DpAXaFXu/2x8YQkctFZJOIbNq7d2/qDjs6Oti9e3d9Z2kYxjgmi4CJ6pkyrv6kqt6kqr2q2jt37lynnS5atKgeczMMI4bJImB2A/NCr48H9jRpLoZhODJZBMwTwCkicqKIdACXAPc2eU6GYaQwKco1+J0l/xjoB3LA36nq9iZPyzCMFCaFgAFQ1QeBB5s9D8Mw3Jk0AsaYiPVJMlodEzCTlLge2IAJGaNlmCxGXqMM65NkTAZMwExSrE+SMRkwFSmFVrVzWJ8kYzJgK5gEWrkftPVJMiYDJmASiLNzXHdf80Nw4npgt8LqyjACTEVKIM6esf9Qib6BoaZfzNYnyWh1bAWTQJI9w7w1hpGOCZgEkuwZ5q0xjHRMwCSwYkkPXYV85HvmrTGMdEzApHDtBaebt8YwqsSMvCkERtRWjIUxjFbHBIwD5q0xjOowFckwjMwwAWMYRmaYgDEMIzNEVdO3mmSIyF7gBYdNXwv8LOPpZIHNu7HYvNM5QVUntPOYkgLGFRHZpKq9zZ5Hpdi8G4vNu3pMRTIMIzNMwBiGkRnTXcDc1OwJVInNu7HYvKtkWttgDMPIlum+gjEMI0NMwBiGkRnTVsCIyNtFZFBEdojI6haYzzwReVREnhKR7SJyhT8+R0QeEpGf+n+7/XERka/4839SRM4M7esyf/ufishlDZh7TkQGROR+//WJIrLRP/56v584IjLDf73Df39BaB9r/PFBEVnegDl3ichdIvK0f87fOknO9ZX+7+PHInK7iMxs6fOtqtPugdff+hngJKAD2Aqc1uQ5HQuc6T9/NfDvwGnA54HV/vhq4HP+83cA3wUEWAps9MfnAM/6f7v9590Zz/1TwD8C9/uv7wAu8Z9/Hfi4//yPgK/7zy8B1vvPT/P/BzOAE/3/TS7jOd8K/IH/vAPoavVzDfQAzwGF0Hn+YCuf76ZdUM18AG8F+kOv1wBrmj2vsjl+B/hdYBA41h87Fhj0n38DuDS0/aD//qXAN0Lj47bLYJ7HAw8D5wD3+xfhz4D28nMN9ANv9Z+3+9tJ+fkPb5fRnF/jX6hSNt7q57oH2OULtHb/fC9v5fM9XVWk4B8VsNsfawn8pewSYCPwelV9EcD/+zp/s7jv0OjvdiPwp8Co//oYYFhVj0Qcf2xu/vsH/O0bPeeTgL3ALb5q900RmUWLn2tVHQK+AOwEXsQ7f5tp4fM9XQWMRIy1hL9eRF4F3A18UlV/kbRpxJgmjNcdEXkn8JKqbnaYV9J7jf5/tANnAl9T1SXAQTyVKI6WmLdvE7oQT605DpgFnJcwh6bPe7oKmN3AvNDr44E9TZrLGCKSxxMut6nqPf7wf4rIsf77xwIv+eNx36GR320ZcIGIPA98G09NuhHoEpGgmFn4+GNz89+fDexr8JyDeexW1Y3+67vwBE4rn2uA3wGeU9W9qloC7gF+gxY+39NVwDwBnOJb3zvwDGD3NnNCIiLAzcBTqvql0Fv3AoF34jI820ww/vu+h2MpcMBf1vcD54pIt3/HO9cfqzuqukZVj1fVBXjn8BFVfT/wKPDemDkH3+W9/vbqj1/iez1OBE4BHs9izv68/wPYJSJBYeXfBn5CC59rn53AUhHp9H8vwbxb93xnZZBq9QeeZ+Df8Szof94C8/lveMvUJ4Et/uMdeDrzw8BP/b9z/O0F+Ko//21Ab2hfHwZ2+I8PNWj+b+OoF+kk/we7A7gTmOGPz/Rf7/DfPyn0+T/3v8sgcF4D5rsY2OSf7z48L1DLn2vgOuBp4MfAP+B5glr2fFuqgGEYmTFdVSTDMBqACRjDMDLDBIxhGJlhAsYwjMwwAWMYRmZYZ0fDCREJXLgAvwaM4IXbA5ylqoebMKd+4L2q+stGH9tww9zURsWIyLXAr1T1C2XjgvebGo38YP2O35DjGLVjKpJREyLyBr82ydeBHwHzRGQ49P4lIvJN//nrReQeEdkkIo/7UbHl+/sDEfknEen3a5VcHXOcY0Vkt4h0+e9/yK/VslVEbnE9npEtpiIZ9eA0vCjWj4VyYqL4CvB5Vd3gZ4zfD7wxYruz/PHDwBPiFbL6Vfg4AN5CBkRkEfBp4DdUdZ+IzKnweEZGmIAx6sEzqvqEw3a/AywMBAPQLSIFVS2WbdevqvsBRKQPL43iewnHOQevmNI+gOBvBcczMsIEjFEPDoaejzK+HMDM0HPBzSBcbhgMXh8s3zC03yhjouvxjIwwG4xRV3zD634ROUVE2oB3h97+AfCJ4IWILI7Zzbni1cztxKt/8ljKYX+Alx08x9/vnNC4y/GMjDABY2TBp/FUmofxao8EfAJY5htjfwJ8NObz/4JX43cAuF1VtyQdTFWfxKun+88isgVYV+HxjIwwN7XRUojIHwBvVNVPNnsuRu3YCsYwjMywFYxhGJlhKxjDMDLDBIxhGJlhAsYwjMwwAWMYRmaYgDEMIzP+P3E0hB1039jJAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 288x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#还可以观察预测值与真值的散点图\n",
    "plt.figure(figsize=(4, 3))\n",
    "plt.scatter(y_train, y_train_pred_lr)\n",
    "plt.plot([-3, 3], [-3, 3], '--k')   #数据已经标准化，3倍标准差即可\n",
    "plt.axis('tight')\n",
    "plt.xlabel('True price')\n",
    "plt.ylabel('Predicted price')\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
